I found the answer, for those wondering, you can define org-agenda-files list within the org-agenda-custom-commands section which limits the scope of the custom agenda to only those files. For reference:
Is it possible to tell orgmode to ignore the contents of an entire file when rendering a custom agenda even if that file is included in my org-agenda-file property?