Ankush Rai

Ankush Rai

Software Engineer Scalable Backends · Distributed Systems · AI / LLM Systems · AWS

Software engineer with 4+ years of experience shipping production systems end-to-end, from high-throughput APIs and event-driven backends on AWS to AI-native products built on top of LLMs, RAG, MCP, and agentic tool-use. I move fast across the stack: backend, frontend, ML, and infra. If it can be built, I'll build it.

Contact Resume GitHub
Portrait of Ankush Rai

Experience

  1. Allyvia - Software Engineer Intern

    Mar 2026 – Present · San Francisco Bay Area (Remote)

    • Built an event-driven client data ingestion pipeline using AWS SQS to decouple third-party sync jobs from the API, enabling reliable retries and back-pressure handling for high-volume QuickBooks/Square webhook traffic.
    • Engineered a multi-tenant Django REST API that integrates with QuickBooks and Square to unify finance, inventory, and operations data into a single hub.
    • Designed responsive frontend interfaces in React and TypeScript to surface integrated business data and AI-driven insights to end users.

  2. ZS - Senior Software Engineer

    Jul 2024 – Jan 2025 · Bengaluru, IN

    • Led migration from a legacy Tableau dashboard to a modern web application using React, FastAPI, and Spring Boot with Redis, reducing latency by 45% and eliminating $100K/year in licensing costs.
    • Built ML microservice for anomaly detection in clinical system logs and operational metrics using isolation forests and time-series features, reducing triage time by 30%.
    • Developed a real-time notification system with WebSockets and AWS MSK, enabling sub-580ms delivery for alerts used by research and operations teams with 99.9% uptime.
    • Implemented automated CI/CD pipelines with GitLab and Docker, replacing ad-hoc scripts and sequential builds - cutting deployment time by 35% and ensuring smoother, more reliable releases.

  3. ZS - Software Engineer

    Mar 2022 – Jun 2024 · Bengaluru, IN

    • Developed a GenAI-powered pipeline to extract KPIs from 10K+ unstructured healthcare reports using prompt chaining and preprocessing - automated 20+ analyst hours/week.
    • Optimised RESTful APIs to handle 3 times higher concurrent load by tuning DB connection pooling, adding Redis caching, and validating stability through JMeter load tests.
    • Integrated Prometheus and Grafana to monitor service health and improve incident response time.
    • Protected HIPAA-compliant pharmaceutical APIs by integrating Okta SSO with JWT tokens and rate-limiting, securing access to 500K+ patient records for research teams.

  4. EY - Software Engineer I

    Oct 2020 – Feb 2022 · Bengaluru, IN

    • Improved backend performance by 35% for a large automotive platform by implementing caching, pagination, and streamlined payload handling across services built with NodeJS and Spring Boot.
    • Fixed race conditions in multithreaded services using locks and task queues - enhanced system stability and reduced transactional errors by 15%.
    • Tuned PostgreSQL performance using indexing and execution plan analysis, cutting query latency in key workflows.
    • Built robust unit and integration test suites with Pytest and Postman, increasing coverage and reducing production issues.

  5. Pantech Solutions - Machine Learning Intern

    Aug 2019 – Nov 2019 · Chennai, IN

    • Developed data validation and preprocessing scripts for real-time stock data pipelines, improving training input quality.
    • Trained and validated ML models (Linear Regression, SVM) using scikit-learn to predict short-term stock trends, applying feature engineering for improved accuracy.
    • Documented edge cases and model drift patterns to improve debugging and retraining workflows.

Education

  1. Santa Clara University

    Santa Clara, CA · March 2025 – March 2027

    Master of Science in Computer Science and Engineering

  2. SRM Institute of Science and Technology

    India · August 2016 – June 2020

    Bachelor of Technology in Computer Science and Engineering

Projects

Personal & Open Source

Argus dashboard screenshot

Argus: MCP Prompt-Injection Firewall

An MCP-native security firewall that sits between AI agents and their tools, inspecting every tool call and response for prompt injection, credential exfiltration, and policy violations. FastAPI proxy with a JWT identity layer, layered detection (regex + Claude-as-judge), and a live websocket dashboard showing per-agent decisions and which defense caught each block. Demos cover poisoned tool responses, outbound credential exfil, and the Claude Code CLAUDE.md exploit pattern. Built for the A10 Networks Hackathon (Track 2).

🛡️ AI Security · Prompt-Injection Defense for MCP Agents

  • Python
  • FastAPI
  • MCP
  • Security
  • Prompt Injection
  • JWT
  • Claude API
  • WebSockets
FormWhisper UI screenshot

FormWhisper: Voice-Driven Government Form Filling

Voice-first AI assistant that fills complex PDF forms (e.g., FEMA Disaster Aid 009-0-3) entirely by speaking. A Qwen2.5-VL-32B vision model extracts every fillable field, ElevenLabs TTS reads questions aloud, self-hosted Whisper transcribes answers, and PyMuPDF writes them back into the original AcroForm. Built at Hack4Humanity to make disaster-relief paperwork accessible to people with limited literacy, vision impairments, or in crisis.

🏆 Winner - Best Future Unicorn @ Hack4Humanity

  • Python
  • FastAPI
  • React
  • Qwen2.5-VL
  • vLLM
  • Whisper
  • ElevenLabs
  • PyMuPDF
PaperTrail screenshot

PaperTrail: NVIDIA AI Document Intelligence for Nonprofits

Real-time document understanding platform for nonprofits. Parses PDFs with pypdf/pypdfium2 + NVIDIA OCR NIM, embeds chunks with NVIDIA embed-qa-4, retrieves via in-memory vector store, and generates structured JSON with NVIDIA Nemotron showing critical/important insights, deadlines, and AI-generated action steps.

  • Python
  • FastAPI
  • NVIDIA NIM
  • RAG
  • LangChain
  • Vector DB
  • OCR
  • Nemotron
CFO Copilot screenshot

CFO Copilot: Mini FP&A Agent

Lightweight FP&A assistant built with Streamlit that reads CSV fixtures and answers common finance questions with text summaries, charts, and 2-page PDF exports. Features rule-based intent classifier with TF-IDF/embedding fallback for Revenue vs Budget, Gross Margin %, Opex, EBITDA, and Cash runway metrics.

  • Python
  • Streamlit
  • RAG
  • sentence-transformers
  • TF‑IDF
  • Pandas
  • PDF Export
cf_ai_chat_memory screenshot

cf_ai_chat_memory: Persistent AI Chat on Cloudflare

A serverless AI chat assistant built entirely on Cloudflare that remembers conversations. Uses Llama 3.1 8B via Workers AI for intelligent responses, Cloudflare Workers for edge compute, and Cloudflare KV for persistent conversation memory with 24-hour sessions.

  • JavaScript
  • Cloudflare Workers
  • Workers AI
  • Llama 3.1 8B
  • Cloudflare KV
  • Serverless
  • Edge Computing
  • Wrangler

Virtual Memory Management Simulator

Simulates OS virtual memory paging using five page replacement algorithms (FIFO, LRU, LFU, MFU, Random). Models locality of reference with 150 random processes, tracking hit ratios and page swaps.

  • C
  • Operating Systems
  • Algorithms
  • Memory Management
  • Data Structures
Acoustic Shield screenshot

Acoustic Shield: AI Vehicle Emergency Sound Detection

AI-powered audio classification system deployed on AWS that detects vehicle emergency sounds in real-time. Uses wav2vec2 model to classify tire skids, emergency braking, and collision-imminent scenarios with sub-500ms latency.

  • Python
  • AWS SageMaker
  • PyTorch
  • HuggingFace
  • wav2vec2
  • Audio ML
  • S3
  • librosa
GreenSense screenshot

GreenSense: Automated Plant Health Monitoring

JavaFX plant monitoring system with event-driven architecture and custom EventBus. Implements environmental simulation engine with automated hydration/thermal regulation, pest detection, and species-parameterized health model using SOLID design patterns (Singleton, Observer, Factory).

  • Java
  • JavaFX
  • Maven
  • OOP
  • Design Patterns
  • Event-Driven

Enterprise & Industry Projects

Digital Control Tower

Amgen · ZS Associates

Led development of a centralized monitoring platform that tracks API performance and system health metrics in real-time; ML-driven anomaly detection reduces incident response time and prevents ~15% of potential downtime events.

  • Java
  • Spring Boot
  • Python
  • Flask
  • scikit‑learn
  • JavaScript
  • ML

Quality Control Schedule Optimization

Pfizer · ZS Associates

Real-time notification system delivering live alerts via Kafka and WebSockets on AWS MSK; achieved sub-580ms latency with 99.9% uptime for critical quality control operations.

  • FastAPI
  • Kafka (MSK)
  • AWS
  • GenAI

Adaptive Onboarding Experience Engine

General Motors · EY

Collaborative filtering recommendation system that personalizes content on user onboarding dashboards, significantly boosting feature adoption and user engagement across GM's digital platforms.

  • Python
  • scikit‑learn
  • Pandas
  • Flask

Skills

Languages & Frameworks

Languages: Python, Java, JavaScript/TypeScript, C

Backend: FastAPI, Spring Boot, Flask, Express.js

Frontend: React, Streamlit

AI / LLM Systems

Agents & Tools: Claude API, MCP, OpenAI API, agentic tool-use, prompt-injection defense

RAG & Retrieval: LangChain, vector DBs, sentence-transformers, NVIDIA NIM, embeddings

Models & Inference: Hugging Face, vLLM, PyTorch, TensorFlow, scikit-learn, Whisper, wav2vec2

Data & Infrastructure

Databases: PostgreSQL, MySQL, Redis, Vector DBs

Streaming: Kafka, AWS MSK, WebSockets

Data Tools: Pandas, NumPy, Matplotlib, Jupyter

Cloud & DevOps

Cloud: AWS (EC2, S3, MSK, Lambda), Azure

DevOps: Docker, GitLab CI/CD, GitHub Actions

Monitoring: Prometheus, Grafana, CloudWatch

Publications

Awards

Contact

Email Me LinkedIn