Tony Chen
I'm a third-year undergraduate studying computer science at Columbia. I'm interested in multimodal learning, especially treating time series as a native modality for language models.
Right now, I'm a research intern with the OpenTSLM team at Stanford. Before that, I worked on ECG-language models at Carnegie Mellon's Safe AI Lab and on plant growth monitoring with Prof. Huiming Yin at Columbia.
news
| Oct 2026 |
|
|---|---|
| Sep 2026 |
|
| Jul 2026 | |
| Jan 2026 |
|
| Sep 2025 |
|
| Jun 2025 |
|
| Aug 2024 |
|
| Apr 2024 |
|
publications
* denotes equal contribution. Also on Google Scholar.
-
NeurIPS WOpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and ReasoningNeurIPS FMTS Workshop, 2026
Real-world time-series applications increasingly require models that can handle time series forecasting, context-conditioned prediction, and language-based temporal reasoning. Yet current time-series foundation models remain fragmented across these capabilities: numerical specialists often provide the strongest forecasts, while language-based models offer broader contextual understanding and analysis. A central challenge is to unify these heterogeneous capabilities without reducing their individual performance. We introduce OpenTSLM TeeMoE, a generalist time-series language model that can forecast directly from observed time series, reason over textual context and temporal patterns, and synthesize and refine predictions from external numerical forecasting specialists. We independently train three low-rank experts for forecast aggregation, native forecasting, and temporal analysis over a shared backbone. A learned LoRA mixture-of-experts controller then weights their frozen parameter updates for each request. Our proposed model achieves strong performance on widely used benchmarks for time series forecasting, context-conditioned prediction, and language-based temporal reasoning, ranking among the top three on GIFT-Eval by mean MASE rank, Context is Key by RCRPS, and TimeSeriesExam by accuracy.
@article{chen2026teemoe, title = {OpenTSLM TeeMoE: A Unified Time-Series Language Model for Forecasting, Contextual Prediction, and Reasoning}, author = {Tony Chen and Timo Stoffregen and Maxwell Xu and Thomas Kaar and Martin Maritsch and Geremia Pompei and Nicolas Zumarraga and Robert Jakob and Paul Schmiedmayer and Patrick Langer and Juncheng Liu}, journal = {arXiv preprint arXiv:2609.40265}, year = {2026}} -
arXivTimeNet: An Extensible Unified Data Infrastructure for Next-Generation Temporal Foundation ModelsarXiv preprint, 2026
Temporal Foundation Models (TFMs) aim to generalize across domains, datasets, and tasks. Yet, their development remains constrained by fragmented, task-specific data formats, annotations, and processing pipelines. We introduce TimeNet, an open-source data standard and scalable infrastructure that decouples temporal data from task definitions and represents signals, metadata, annotations, and supervision in a shared, extensible data model. TimeNet supports multimodal signals with regular, irregular, or ordinal time axes and expresses different task families (including classification, forecasting, temporal localization, question answering, generation, and editing) as reusable views over the same recordings. This shared representation enables heterogeneous time-series datasets to be combined for large-scale model training across domains, modalities, and tasks. We demonstrate TimeNet by transcoding datasets with 1.5M task instances spanning diverse domains, modalities, temporal scales, and forms of supervision, while retaining practical I/O performance relative to native formats. TimeNet enables an existing TFM training pipeline to support joint training on a configurable number of heterogeneous datasets through configuration changes alone. We show this capability by training TFM across multiple datasets and tasks, obtaining a 14% F1 score improvement compared with models trained on individual datasets. These results show that TimeNet provides the data and systems foundation needed to move beyond task- and dataset-specific TFMs toward models that can learn jointly across heterogeneous domains, modalities, temporal scales, and forms of supervision from a common data model.
@article{maritsch2026timenet, title = {TimeNet: An Extensible Unified Data Infrastructure for Next-Generation Temporal Foundation Models}, author = {Martin Maritsch and Timo Stoffregen and Thomas Kaar and Behsad Riemer and Maxwell A. Xu and Max Rosenblattl and Juncheng Liu and Nicolas Zumarraga and Yu Yvonne Wu and Denys Herasymuk and Sparsh Rastogi and Hyungjun Yoon and Bosong Huang and Arvind Pillai and Dmytro Lopushanskyy and Tony Chen and Robin Deuber and Yichen Liu and Shvat Messica and Dan Li and Jian Lou and Yuwei Zhang and Jaeho Kim and Renée Rosillo Garcia and Fan Wu and Robert Müller and Elgar Fleisch and Flora D. Salim and Dimitris Spathis and Yuzhe Yang and Aaqib Saeed and Daniel McDuff and Ming Jin and Markus Kreft and Kevin O'Sullivan and Robert Jakob and Azul Garza and Paul Schmiedmayer and Patrick Langer}, journal = {arXiv preprint arXiv:2610.04407}, year = {2026}} -
MLHCELF: A Family of Encoder-Free ECG-Language ModelsMachine Learning for Healthcare (MLHC), 2026
ECG-Language Models (ELMs) extend recent advances in Multimodal Large Language Models (MLLMs) to automated ECG interpretation. However, most existing ELMs inherit Vision-Language Model (VLM) design choices and rely on pretrained ECG encoders, introducing substantial architectural and training complexity. Inspired by encoder-free VLMs, we introduce ELF, a family of three encoder-free ELMs that remain competitive with, and often outperform, prior state-of-the-art ELMs across two datasets despite substantially simpler architectures and training pipelines. All code and data are available at github.com/ELM-Research/ECG-Language-Models.
@inproceedings{han2026elf, title = {ELF: A Family of Encoder-Free ECG-Language Models}, author = {William Han and Tony Chen and Chaojing Duan and Xiaoyu Song and Yihang Yao and Yuzhe Yang and Michael A. Rosenberg and Emerson Liu and Ding Zhao}, booktitle = {Machine Learning for Healthcare Conference (MLHC)}, year = {2026}} -
arXivRetrieval-Augmented Generation for Electrocardiogram-Language ModelsarXiv preprint, 2025
Interest in generative Electrocardiogram-Language Models (ELMs) is growing, as they can produce textual responses conditioned on ECG signals and textual queries. Unlike traditional classifiers that output label probabilities, ELMs are more versatile, supporting domain-specific tasks (e.g., waveform analysis, diagnosis, prognosis) as well as general tasks (e.g., open-ended questions, dialogue). Retrieval-Augmented Generation (RAG), widely used in Large Language Models (LLMs) to ground LLM outputs in retrieved knowledge, helps reduce hallucinations and improve natural language generation (NLG). However, despite its promise, no open-source implementation or systematic study of RAG pipeline design for ELMs currently exists. To address this gap, we present the first open-source RAG pipeline for ELMs, along with baselines and ablation studies for NLG. Experiments on three public datasets show that ELMs with RAG consistently improves performance over non-RAG baselines and highlights key ELM design considerations.
@article{song2025rag, title = {Retrieval-Augmented Generation for Electrocardiogram-Language Models}, author = {Xiaoyu Song and William Han and Tony Chen and Chaojing Duan and Michael A. Rosenberg and Emerson Liu and Ding Zhao}, journal = {arXiv preprint arXiv:2510.00261}, year = {2025}} -
SmartAgTechCamera-Based Plant Growth Monitoring for Automated Plant Cultivation with Controlled Environment AgricultureSmart Agricultural Technology, 2024
Building integrated photovoltaic thermal (BIPVT) greenhouse technology exhibits great potential to enhance crop yielding, energy harvesting, and water management for highly efficient farming and enables controlled environment agriculture (CEA) for automated plant cultivation. The proposed smart greenhouse system uses BIPVT, geothermal, image-based sensing, and control technologies for high energy efficiency and high-fidelity plant growth control. A camera-based, computer-controlled system was demonstrated to successfully monitor single and multiple food plants' growth. The growth rate over the life-cycle of the plant was automatically recorded by cameras without human interference. A novel algorithm was developed to extract the plants from the background of the image and then to measure the length from the ground to the top of the longest plant with different shapes or profiles. The growth rate was automatically generated as fundamental data for farming management and control. Because the plant growth is slow and the errors caused by image noise can be comparable with the actual growth when the interval between images is not large enough, the noise of a camera is evaluated by comparing multiple images under the same exposure condition, which is used to provide the error range of the measurements. The plant growth monitoring system will be integrated into an ongoing greenhouse project for smart greenhouse operation.
@article{chen2024camera, title = {Camera-Based Plant Growth Monitoring for Automated Plant Cultivation with Controlled Environment Agriculture}, author = {Tony Chen and Huiming Yin}, journal = {Smart Agricultural Technology}, volume = {8}, pages = {100449}, publisher = {Elsevier}, doi = {10.1016/j.atech.2024.100449}, year = {2024}}
projects
-
DatasetCaptchaSolve30kThe largest open-source dataset of human mouse-movement and captcha-solving trajectories: 30,000 verified sessions from thousands of users across three puzzle types, with full behavioral replays.Hugging Face Datasets, 2026 · Apache 2.0
- I led the creation of the dataset and designed the end-to-end collection and verification pipeline.
- Each session captures mouse/touch trajectories, timing, and puzzle state at physics-tick resolution (240 Hz physics, 1000 Hz raw input), and can be replayed deterministically.
- Three puzzle types: Sheep Herding (draw paths to guide sheep into a pen), Thread the Needle (navigate through gaps without touching walls), and Polygon Stacking (drag and stack shapes on a platform).
- Privacy-preserving by design: coordinates are normalized to a 200×200 grid, render sizes are randomized per session, and no IPs, cookies, or persistent identifiers are stored.
- Using CaptchaSolve30k in a paper? I'd love to hear about it, so please email me.