As an undergrad working on my final year project, I’m facing a familiar dilemma: how to achieve cutting-edge AI capabilities on a tight budget. My goals are ambitious: I want to run object detection on medical images and perform visual question answering (VQA) using a small language model, all on-device and without relying on the cloud.
The challenge is finding the right hardware that can handle these demands without breaking the bank. I’ve been eyeing the Jetson boards – specifically the Nano, Orin Nano, and Orin NX – but I’m not sure which one can realistically handle a quantized detector and a small language model for VQA.
## The Requirements
To recap, I need a device that can:
- Run object detection on medical images (PNGs)
- Perform visual question answering with a small LLaMA model
- Do it all on-device, without relying on cloud services
## The Budget Constraints
As a student, my budget is limited, and I need to find the best balance of cost and capability. I’m willing to make some compromises, but I still want to achieve decent performance.
## The Options
The Jetson boards seem like a promising option, but I’m not sure which one is the best fit. Here are my thoughts so far:
- The Jetson Nano is the most affordable option, but will it be able to handle the computational demands of object detection and VQA?
- The Orin Nano and Orin NX offer more powerful processors, but are they worth the extra cost?
## Have You Tried This?
If you’ve worked on a similar project, I’d love to hear your advice. Which hardware would you recommend for this specific use case? What compromises did you have to make, and what were the results?
Thanks for reading, and I look forward to your input!