Hi, I am Jaewook

PhD candidate in Computer Science at the University of Massachusetts Amherst specializing in LLM post-training, evaluation, agentic RAG, and conversational AI. Builds and evaluates language-model systems for chat-based interaction, personalization, user simulation, retrieval, and human-AI collaboration, using methods such as reinforcement learning, tool use, activation steering, and human-centered evaluation. Industry experience includes multimodal search at Apple and evaluation methodology for production multi-agent security agents at AWS. Author of 10+ publications at venues including ACL, EMNLP, NAACL, and CHI.

Technical Skills

Core: LLM Post-Training, Reinforcement Learning, Supervised Fine-tuning, Preference Optimization, LLM Evaluation, Agentic RAG, Personalization, Dialogue Systems, User Simulation, Vision-Language Model
Systems: Multi-Turn Conversational AI, Retrieval Systems, Tool Use, Multi-Agent Systems, Large Language Models, Vision-Language Models, vLLM, RAG Accelerators, AI Accelerators
Evaluation & Alignment: Human Evaluation, Benchmark Design, Evaluator Design, Reward Modeling
Engineering: Python, PyTorch, Hugging Face, iOS, React, Go

Experience

Apple

Machine Learning Engineer Intern · Santa Clara, CA
Jun. 2026 – Aug. 2026

Contributed to multimodal search systems by fine-tuning VLM for Apple's Answers, Knowledge, and Information team; additional technical details limited by confidentiality.

Amazon Web Services

Applied Scientist Intern · Seattle, WA
May 2025 – Aug. 2025

Designed the evaluation methodology for a production multi-agent AppSec framework, defining task rubrics and assessments for design reviews.

Eedi

Machine Learning Research Intern · Remote, London, UK
Jun. 2023 – Aug. 2023

Prototyped AnSearch, an LLM-based diagnostic-generation system that combines model outputs with domain expertise to create targeted assessments for identifying student misconceptions.

Selected Research

Agentic RAG, Tool Use & Retrieval Systems

Developing reinforcement learning and on-policy self-distillation methods to improve spatiotemporal reasoning in VLMs for long-video question answering. (Ongoing project)
Built an agentic RAG policy that trains LLMs with reinforcement learning to choose retrieval granularity and coordinate keyword/semantic search tools (EMNLP 2026).
Evaluated latency–accuracy trade-offs in software–hardware co-designed RAG, analyzing how approximate vs. exact nearest-neighbor retrieval affects time-to-first-token and downstream answer quality (Submitted to HPCA 2027).

Post-Training, Dialogue Systems & Model Behavior

Developing tool-using LLM agents that simulate students by editing executable (SymPy) solution programs to reproduce each student’s reasoning errors, to predict their future mistakes (Ongoing project).
Built an evaluation framework for LLM-simulated users in tutoring dialogues, measuring linguistic, behavioral, and cognitive realism and showing that surface-level plausibility can mask failures in learning dynamics (ACL 2026).
Developed a preference-optimization method for deriving tutor-persona steering vectors from human tutor-student dialogues, enabling interpretable control of LLM dialogue behavior without explicit persona prompting (ACL 2026 BEA Workshop).
Developed an Expectation Maximization-based personalization framework that infers community-level and user-specific latent preferences, enabling LLM generation under cold-start conditions (EMNLP 2025).

Applied NLP, Personalization & Human-Centered Evaluation

Built an iOS prototype for adaptive LLM generation that personalizes learning content to individual user preferences through interactive feedback (CHI EA 2026).
Led research on phonological similarity algorithms for cross-lingual keyword discovery grounded in user language priors, supporting personalized mnemonic generation (EMNLP 2025).
Designed evaluation pipelines combining psycholinguistic metrics with human assessment to measure memorability and creative quality in LLM-generated mnemonics (EMNLP 2024).

Education

University of Massachusetts Amherst

Ph.D. in Computer Science · Sep. 2022 – Present
Amherst, MA

Advisor: Dr. Andrew Lan
Research: LLM Post-Training and Evaluation, Personalization, Human-in-the-Loop AI
Mentored student-led ML/NLP research projects, including collaborations with Spotify and Adobe Research
Passed qualifying exam with distinction (May 2025); M.S. earned Feb. 2026

Korea University

M.E. & B.E. in Electrical and Computer Engineering · Mar. 2013 – Feb. 2022
Seoul, South Korea

M.E. (Jan. 2019 – Feb. 2022), Advisor: Dr. Seon Wook Kim
Research: Compiler Optimization, Processing-in-Memory, AI Frameworks
Developed ONNX Runtime integration and profiling/scheduling algorithms for PIM-based DNN inference with SK Hynix
Built NPU compiler tooling for memory-trace extraction and scratchpad-aware LLVM code generation with Samsung
B.E. (Mar. 2013 – Aug. 2019), Graduated with honors; Mandatory Military Service (21 months)

Awards

2023 – NAEP Math Automated Scoring Challenge Grand Prize

2022 – NeurIPS 2022 Causal Edu Competition (Task 3), 3rd Place

2018 – iOS Application Hackathon Grand Prize, Korea University

My journey so far

“Used to map matmuls to silicon. Now I wrap LLMs in uncertainty—and snap when the app shows a gap.”

It’s a playful rhyme for my journey: from studying electrical engineering in my bachelor’s to exploring large language models in education for my Ph.D.

Prototype of Vocabulary Builder

Back in 2018—pre-ChatGPT era—I participated in an iOS Hackathon and built a vocabulary-builder prototype that won the grand prize. My vision for future work was to automatically generate keyword mnemonics and pair them with images combining those keywords. At the time, automatically generating keyword mnemonics was impossible, so I instead pursued a master’s in computer engineering to build strong foundations in both hardware and software.

Process-in-Memory Emulator

During my master’s, I worked extensively with FPGA platforms. A notable project was developing a software interface for Processing-in-Memory (PIM) emulator, which became the basis for my first journal publication. My role involved creating an interface between the PIM hardware and ONNX Runtime, then designing a scheduling algorithm to optimize execution on a heterogeneous platform. Because the device had no internet access, the only way to transfer the framework was via an SD card—a process I repeated so many times that the card eventually wore out and had to be replaced.

From this experience, I discovered two things:

I love writing papers—it feels like wrapping a hard-earned gift.

I finally grasped the value of a top-down approach in research, after struggling early in my master’s by treating research like coursework.

I chose a Ph.D. in my current field because I wanted to make education more engaging. The vocabulary-builder project remains one of my happiest moments—coding all night felt like pure fun. When I started my Ph.D., the sudden boom of LLMs made possible what I had once only imagined: automatic mnemonic generation. This became the topic of my first publication.

Now, I work on applying machine learning, reinforcement learning, and evaluation methods for LLM-generated content, as well as developing web apps for conducting human evaluations.

Natural language processing · ML systems

Hi, I’m Jaewook.

I study how language models behave—and build systems that make them useful and efficient.

I’m a Computer Science Ph.D. candidate at UMass Amherst, advised by Andrew Lan. My work connects two perspectives: NLP research on post-training, evaluation, and personalization; and systems research on agentic retrieval, compilers, and hardware-aware inference.

View CV Google Scholar

Research & industryUMass Amherst · Ph.D.Apple · ML engineering internAWS · Applied scientist intern

01 / Natural language processing

Understanding and shaping model behavior.

All publications on Scholar

NLP × Systems · Agentic retrieval

Learning when to search,
read, and answer.

GRASP trains language-model agents to coordinate semantic search, keyword search, and paragraph reading. The policy learns how to acquire evidence and when to expand its context.

Explore GRASP Project & code

01Search broadly
Find clues with semantic retrieval

02Read selectively
Expand a sentence’s parent paragraph

03Search precisely
Resolve entities with keyword retrieval

An illustration of the available actions. The learned policy chooses their order and when to answer.

Evaluation · User simulation

Beyond a convincing conversation.

When do LLM-simulated students behave like real learners? We evaluate linguistic, behavioral, and cognitive realism, showing why surface-level plausibility can hide failures in learning dynamics.

Substance or Illusion?ACL 2026

Post-training · Activation steering

Learning behavior from real dialogue.

Tutor-persona steering vectors learned from human tutor–student dialogues offer interpretable control over model behavior. Preference optimization connects observed conversations to how a model responds.

Letting Tutor Personas Speak UpBEA 2026 · First author

Personalization · Human-centered AI

A memory aid that fits the learner.

From interpretable preference modeling to an interactive iOS prototype, this work explores how language models can generate kanji mnemonics that adapt to individual learners—even with limited feedback.

EM personalization Adaptive mnemonic designEMNLP 2025 · CHI EA 2026

A closer look / GRASP

What should an agent read next?

Multi-hop questions require more than retrieving a relevant document. An agent must identify the next missing piece of evidence, choose a useful tool, and decide how much context to expose.

GRASP uses GRPO to learn these decisions. Search returns sentences; a separate read action expands their parent paragraphs. The training signal combines answer accuracy, grounded reading, complementary search, and turn efficiency.

The updated study evaluates both 3B and 7B models on HotpotQA, 2WikiMultihopQA, and MuSiQue.

Explore the project

72 F1

HotpotQA · 7B model · 500 sampled validation questions

GRASP — 72

AutoRefine — 65

Token F1 × 100. Strongest listed baseline on this metric: AutoRefine. Updated manuscript, Table 3. GRASP ties AutoRefine on MuSiQue F1.

What remains open

The training rewards rely on gold supporting-document annotations. Learning grounded tool use with weaker supervision is an important next step.

02 / ML Systems

Making inference work efficiently.

The model is one part of the system. Retrieval, data movement, and where computation runs shape its practical performance.

Hardware–software co-design · Inference

Choosing where a model should run.

Model partitioning and low-overhead profiling for deep-learning inference on a processing-in-memory platform. This work connects runtime decisions to the capabilities of heterogeneous hardware.

Model partitioningACM TODAES · Co-first author

ML infrastructure · Runtime integration

Bringing PIM into ONNX Runtime.

An execution provider connects ONNX models to processing-in-memory operations. The framework offloads supported operators and evaluates end-to-end BERT inference on an FPGA-emulated PIM platform.

ONNX Runtime paperICEIC 2022

Compilers · Memory management

Generating code around the memory hierarchy.

Extending LLVM/Polly to generate tiled execution, memory transfers, and address calculations for scratchpad memory. The study examines how tile shapes and reuse affect data movement.

Tile-based code generationICEIC 2021 · First author

Connecting NLP and systems

GRASP studies retrieval decisions through learned policies. My ongoing RAG systems work also examines the latency–accuracy trade-offs of approximate versus exact retrieval, including time-to-first-token and downstream answer quality.

Current research in the CV

03 / Research notes

Questions behind the work.

On retrieval

Finding evidence is only the beginning.

A revealing GRASP example asks about an actor’s role in another film. Several agents find sufficient evidence yet answer with the actor’s name instead of the character. The example highlights a distinction between finding evidence and preserving the relationship the question asks about.

Read the research context

On evaluation

What does “realistic” actually measure?

A simulated learner can sound natural while failing to reproduce how people learn. Evaluating language, behavior, and cognition separately makes those gaps visible. The useful question becomes which properties of a real user a simulation needs to preserve for a particular experiment.

Read the study

04 / A little background

From computer memory
to human memory.

In 2018, I built an iOS vocabulary-learning prototype at a hackathon. I wanted to automatically generate keyword mnemonics and images that could make unfamiliar words easier to remember. That question stayed with me.

During my master’s at Korea University, I worked on compilers, scratchpad memory, and processing-in-memory systems. My Ph.D. brought me back to learning—this time with language models that made automatic mnemonic generation possible.

Today, that background informs my work across model behavior, evaluation, personalized generation, and retrieval. I’ve also worked on multimodal search at Apple and evaluation methodology for security agents at AWS.

Experience & education

A systems foundation

LLVM/Polly code generation, ONNX Runtime integration, and model partitioning for processing-in-memory inference.

Compiler work Runtime work Model partitioning

My research across the AI stack

Human-AI interaction
Personalization · Tutoring · Human evaluation
Models & learning
LLM post-training · Reinforcement learning
Retrieval & inference
Agentic RAG · Latency-accuracy trade-offs
Compilers & runtimes
LLVM · ONNX Runtime · Scheduling
Hardware-aware computing
Processing-in-memory · AI accelerators

Education

Industry (Intern)