Tomofun deploys vision-language models on AWS Inferentia2 chips to reduce inference costs for real-time pet behavior detection

Tomofun, a Taiwan-headquartered pet-tech startup behind the Furbo Pet Camera, migrated its pet behavior detection inference workloads from GPU-based Amazon EC2 instances to EC2 Inf2 instances powered by AWS Inferentia2 to reduce costs while maintaining accuracy for always-on monitoring across hundreds of thousands of devices.

The BLIP vision-language model (a model that interprets images and generates text descriptions) was decomposed into three components—Image Encoder, Text Encoder, and Text Decoder—each compiled independently using torch_neuronx and combined into the inference pipeline without altering the original pretrained logic, using lightweight wrapper classes to standardize inputs and outputs.

The architecture uses Elastic Load Balancing and EC2 Auto Scaling groups across two layers: one for API servers and one dedicated to model inference on Inf2 instances, with Amazon CloudWatch monitoring latency, throughput, and error rates to maintain service-level objectives as demand shifts.