IBM
May 2026 → Aug 2026
Software Developer Intern / San Jose, California
Worked inside an internal LLM evaluation platform that stress-tests IBM's AI coding assistants on mainframe modernization tasks: code explanation, COBOL to Java transformation, and generation.
-
01
Extended the benchmarking pipeline builder, pairing a Python and Streamlit wizard flow with an evaluation service over HTTP.
-
02
Operationalized an LLM-as-a-Judge layer that grades generated code through rubric scoring, pairwise A/B comparison and risk assessment, replacing subjective manual review with reproducible grading at thousands of samples per run.
-
03
Unified two assistant surfaces behind one benchmarking flow, refactoring roughly 3,000+ lines of near-duplicated logic into a single shared module.
-
04
Root-caused an out-of-memory crash under load with a reproducible harness that ramps concurrency while sampling process memory in real time, isolating it to unbounded query deserialization, then re-architected the hottest endpoints to bound resident memory.
5.5×
Less peak memory
6.0 → 1.1
Gigabytes resident
100 → 0
Percent error rate
4 → 512
Concurrent users