That is a nice feature I had forgotten about!

When I export your snippet to html I get this:
<p>
This file was exported on .
</p>

<p>
The answer to 2 + 3 is .</p>
</div>

Is there some trick to make them evaluate? Probably I have set a variable in org-mode to not evaluate code on export, because I usually do not want that.

I could type C-cC-c on the src, which puts the result next to it like =5=, which then is exported to the html.

The links are clickable, show the output in the minibuffer, and are evaluated at export time.  These are two slightly different ways to solve the problem. Also, you could do fancier things with the filter approach like make tooltips with the original code snippet, etc...