Yes, tables and source blocks handle captions in the HTML exporter, but
special blocks (which #+(BEGIN|END)_FIGURE is) don't. Captions for special
blocks *are* handled correctly in the latex exporter. This shouldn't be
too hard to fix: just copy the caption code from org-html-table to
org-html-special-block, but I won't have time to try it today. And then
there is the question of other exporters...