
## Project Overview

I originally built this for a research project where we needed to systematically evaluate how well different language models could predict sentiment ratings for a massive dataset of Amazon reviews.

Turns out, getting consistent, comparable results across dozens of different LLM providers is trickier than it sounds. Each has different APIs, rate limits, response formats, and quirks.

Along the way, I added support for vision models because, honestly, I was curious whether multimodal LLMs could rate meme sentiment as well as they handle text reviews. Spoiler: they can, and it's pretty entertaining.

---

## 🎯 The Problem

Evaluating LLMs on sentiment tasks sounds straightforward until you want to test more than just OpenAI models. You end up dealing with:

- **Provider chaos**: Every company has different API formats, authentication, and rate limits
- **Local vs cloud**: Completely different connection patterns for hosted vs self-served models
- **Result tracking**: Managing experiments across dozens of model combinations
- **Batch processing**: Staying efficient while respecting each provider's constraints

---

## 🔧 Technical Design

**LiteLLM as the Universal Adapter**

I nearly cried with joy when I found out that LiteLLM exists, I initially thought I might have to do all the heavy lifting here. Instead of writing separate clients for every provider, I used LiteLLM as a universal translation layer. It handles the API differences between OpenAI's format, Anthropic's format, Google's format, and 100+ other providers, then converts everything to a consistent response structure.

```python
# Same code works for any provider
result = await model_manager.evaluate_single(
    content="Great product!",
    model_name="gpt-4"  # or "claude-3-haiku", "gemini-pro", "mistral-large"
)
```

**Async-First Architecture**

Everything is built around async/await from the ground up. The ModelManager handles concurrent requests with proper rate limiting and exponential backoff. The Evaluator processes batches in parallel while respecting each provider's API constraints. This made the difference between hour-long evaluation runs and minutes.

**vLLM Integration**

For local models, I integrated with vLLM's OpenAI-compatible server. This is very convenient. I can spin up any Hugging Face model on my GPU and evaluate it using the exact same code path as cloud providers.

**Multi-modal Pipeline**

Adding vision support meant rethinking the data flow. The DataProcessor now handles both text content and image paths. Vision models get base64-encoded images, while text models get plain strings. The evaluation metrics work the same either way.

---

## 📊 Results & Learnings

**Performance**

With proper async batching, I can evaluate 1000 reviews in under 2 minutes using most cloud providers. Local models with vLLM are even faster if you have the GPU memory, plus you're not paying per token.

**Model Insights**

The largest and most recent models consistently outperform smaller models on sentiment accuracy. Recent smaller models like Llama-3 7B do surprisingly well on obvious cases, but struggle with sarcasm and nuanced sentiment.

**Vision Models**

This was the fun experiment—can vision models rate meme sentiment? GPT-4.1 and Claude-4  do quite well, especially on clearly positive/negative visual content. 

---

## 🧠 What I Learned

- **Provider Abstraction**: LiteLLM's universal API is genuinely useful for research code—it eliminated hundreds of lines of provider-specific logic
- **Async Python**: Building everything async-first from the start saves massive refactoring later
- **Model Serving**: vLLM's OpenAI compatibility makes local deployment much simpler than custom inference servers
- **Evaluation Methodology**: Consistent prompting and result tracking matter more than I thought, especially when comparing across providers
- **Infrastructure**: Running evaluations at scale requires thinking about rate limits, retries, and result persistence for every provider

This project scratched a specific research itch, but it ended up being a solid foundation for any LLM evaluation work across the entire ecosystem. The modular design means I can easily add new evaluation metrics or data formats as LiteLLM adds support for new providers.

Plus, I learned that building good research tools that work across the whole LLM landscape is almost as satisfying as the research itself.

A Python library for evaluating LLM performance on sentiment analysis tasks. Supports text and image inputs with async processing and comprehensive metrics.

LLM Sentiment Analyzer

## Project Overview

In the late 19th and early 20th centuries, printers produced stunning hand-colored travel images known as *photochroms*. These prints—somewhere between a photograph and a painting—offered a colorful look at the world decades before color photography was widely available. Skilled artisans added color to black-and-white photos using lithographic stones, crafting a distinct visual style that was both realistic and dreamlike.

I’ve always loved the look of these images. So I started wondering: could a neural network learn to replicate that aesthetic?

This project became my excuse to dive into computer vision and get hands-on with my university’s high-performance computing cluster. Plus, the Library of Congress has digitized nearly 6,000 of these photochroms—perfect training data.

---

## 🧱 Stack

- **Architecture**: U-Net encoder-decoder with skip connections
- **Framework**: PyTorch Lightning for scalable, clean training loops
- **Configuration**: Hydra for modular experiment configs
- **Tracking**: Weights & Biases for metrics and visual logging
- **Compute**: SLURM job scripts for multi-node HPC training
- **Color Space**: LAB (not RGB) for perceptual accuracy

---

## 🎨 The Dataset

I used the [Library of Congress Photochrom Collection](https://www.loc.gov/pictures/collection/pgz/), which includes high-resolution scans from the Photoglob Company of Zürich and the Detroit Publishing Company. These aren’t amateur colorizations—they were professional, carefully composed prints.

The collection spans:
- Over 6,000 scenes from Europe and the Middle East
- 500+ images from North America
- Alpine landscapes, city centers, and rural life

---

## 🔬 Technical Approach

**U-Net Backbone**  
I used a classic U-Net architecture: an encoder compresses the input grayscale image into feature maps, while a decoder reconstructs a color version. Skip connections preserve detail that would otherwise be lost in the bottleneck.

**LAB Color Space**  
Rather than predicting RGB directly, the model works in LAB space:
- Takes the L (lightness) channel as input
- Predicts the A and B (color) channels
- Recombines them into a full-color image

This improves perceptual consistency and gives the colors a softer, more natural look.

**Training Pipeline**  
PyTorch Lightning abstracted away most of the distributed training complexity. Hydra handled configuration variants cleanly. Weights & Biases logged metrics and sample outputs. It was smooth to scale up and iterate quickly.

---

## 🖥️ HPC Training

This was my first serious experience using BYU's supercomputer. I trained across 4 nodes with 8 NVIDIA H100s each, which meant I had to:

- Write efficient SLURM job scripts
- Think about data pipelines and I/O bottlenecks
- Schedule jobs and resume training from checkpoints
- Monitor everything remotely

Learning the HPC ecosystem gave me a whole new appreciation for real-world ML deployment.

---

## 📊 Results

![Example output - good photochrom style](/images/good_photochrom.png)
![Example output - decent photochrom style](/images/decent_photochrom.png)
![Example output - inaccurate photochrom style](/images/inaccurate_photochrom.png)

Where it *shines*:
- **Landscapes** – hills, lakes, valleys
- **Architecture** – buildings, bridges, street scenes
- **Simpler compositions** – where there's a clear subject/background

Where it *struggles*:
- Fine detail (e.g. faces, small patterns)
- Dense or chaotic scenes
- Out-of-domain objects that don’t resemble the training data

That said, the model consistently outputs images that feel like photochroms—slightly oversaturated, soft-edged, and nostalgic.

---

## 🧠 What I Learned

- **Vision**: How convolutional architectures handle translation tasks like grayscale-to-color
- **Color**: Why LAB space is often superior to RGB for this kind of task
- **HPC**: Everything from job queuing to distributed training on SLURM
- **Reproducibility**: Modular configs, logging, and tracking make iteration easier
- **History**: Honestly, a lot about lithographic printing and early photographic methods

This project isn’t production-ready, but it’s been an interesting technical challenge.

A U-Net neural network that colorizes black-and-white photos in the style of 1890s photochrom prints. Trained on Library of Congress archives using university HPC resources.

PhotochromAI


## Project Overview

Style Canvas is a web app I built to host and run AI style transfer models. A friend reached out with some models he'd trained and wanted a site where users could try them out. It turned into a fun systems project — build a way to serve AI inference jobs on demand and do it fast, scalable and **CHEAP**.

The models aren’t mine (and not really to my taste), but the infrastructure is solid. When I want to serve my own models, I can plug them right in.

---

## 🧱 Stack

- **Frontend**: `React + TypeScript`
- **Backend**: `NestJS`
- **Infra**: `AWS EC2 (GPU)`, `Cognito`, `DynamoDB`, `Stripe`
- **Model server**: Python running on-demand on a GPU instance

---

## 🔐 JWT Auth with Cognito

I used AWS Cognito for account management and session auth. When a user logs in, Cognito returns a JWT that gets stored in `localStorage` and attached to every request.

The backend verifies the token and sets the user's privileges using Cognito’s JWKS and Passport’s `JwtStrategy`:

```ts
super({
  jwtFromRequest: ExtractJwt.fromAuthHeaderAsBearerToken(),
  secretOrKeyProvider: passportJwtSecret({
    jwksUri: `${COGNITO_DOMAIN}/.well-known/jwks.json`,
  }),
  audience: COGNITO_CLIENT_ID,
  issuer: COGNITO_ISSUER,
  algorithms: ['RS256'],
});
```

> Getting this flow working meant learning about JWTs — how they’re signed, verified, and decoded. Debugging it all gave me a solid understanding of a real world auth tool.

---

## ⚙️ On-Demand EC2 Inference

I initially containerized the models and got `Sagemaker` configured, but I quickly realized how expensive this option was going to be. I decided on a simpler method: when a user submits an image, I **spin up a GPU EC2 instance**, send the job to it, and shut it down when it's idle. This keeps costs near zero when no one's using the app and has a surprisingly low cold boot up period (10-15 seconds).

```ts
await exec(`aws ec2 start-instances --instance-ids ${ML_INSTANCE_ID}`);
await this.waitForServerReady();
const ip = await this.getServerIP();
```

Jobs are processed sequentially through a queue. The backend handles EC2 orchestration, checks token balances, and forwards the image to the model server:

```ts
const response = await fetch(`http://${ip}:5000/generate/image`, {
  method: 'POST',
  body: formData,
  headers: formData.getHeaders(),
});
```
---

## 💳 Stripe & Token System

Each user gets some free tokens on signup. Every time they stylize an image, one token is used. If they run out, they can:

- Buy more tokens (one-time purchase)
- Subscribe for unlimited monthly use

Prices are configured in Stripe with metadata that maps to token counts:

```ts
const lineItems = await stripe.checkout.sessions.listLineItems(session.id);
const price = lineItems.data[0].price;
const tokens = parseInt(price.metadata.tokens, 10);
await this.tokensService.adjustTokens(userId, tokens);
```

The backend verifies webhooks to confirm payments:

```ts
event = stripe.webhooks.constructEvent(payload, signature, STRIPE_SECRET);
```

> This was my first time using Stripe webhooks. They're clean, but I had to learn how to bypass body parsing in NestJS just for that one endpoint.

---

## 🖼️ Frontend

The React frontend handles:

- Auth state and JWT storage (via custom `AuthContext`)
- Image upload and model selection
- Polling job status every few seconds
- Showing styled results once they’re ready

```ts
const token = localStorage.getItem('accessToken');
if (token) {
  fetchUserProfile(); // hit /me with the token
}
```

When inference starts, the UI updates with job progress. If the EC2 instance needs to boot, it shows “Warming up…” until the backend is ready.

> Not fancy, but it works. Uploads feel snappy, the polling is reliable, and the state is reactive across login, logout, and retries.

---

## 🧠 What I Learned

- How JWTs actually work — signature validation, token lifetimes, and decoding with JWKS
- How to control EC2 instances via code and design on-demand compute flows
- How to process queued jobs and coordinate with an ML worker over HTTP
- How to use Stripe Checkout and webhooks to grant in-app credits
- How to structure and build a full-stack system that feels cohesive and responsive

---
But overall, it works. When someone uploads a photo and picks a style, a GPU spins up, runs inference, returns the result, and shuts down when it's done. And that feels pretty cool.


A full-stack platform for hosting AI models with on-demand GPU inference, JWT auth, and a simple token system. Built to be fast, scalable, and cheap.

|

LLM Sentiment Analyzer

PhotochromAI

Style Canvas