#+caption: My caption
#+label: fig:MyFigure
#+begin_src R :exports results :results output graphics :file MyFigure.pdf :width 4.8 :height 6.5 :pointsize 7
plot(1:5,1:5)
#+end_src

The figure is produced and integrated in the text but

#+CAPTION: line should be added to the output and not to the code block. As follows.

#+CAPTION: My caption.

[[file:MyFigure.pdf]]

That is, you evaluate the code block, and once the output is created, add caption to it.

Vikas