🫠 About Me

Hi, I’m Zeyu Han, an undergraduate student at Sichuan University, majoring in software engineering. Recently I’m serving as a research intern under the guidance of Saiqian Zhang at New York University. Before that, I also worked with Huaizu Jiang at Northeastern University, Dongkuan Xu at NC State University, and Yan Wang at Sichuan University.

Ph.D. Aspirations

I’m looking for a Ph.D. position for Fall 2024. Wishing for good fortune on this academic journey!

Research Interests

Standing at the crossroads of the rapid growth of large foundation models (LLMs, diffusion models, vision-language models etc.), my research aspiration is to explore their remarkable potential, and develop advanced foundation models as well as build solid applications leveraging these models which can democratize to serve a broader range of people. Additionally, I’m also highly interested in reducing the computational cost (training/inference speed, memory/energy cost) of the large models from various aspects (data-level, model-level, system-level) while maintaining or even surpassing the current performance.

Currently, I am focusing on but not limited to these research topics:

• DNN acceleration (PEFT methods)
• Visual grounding
• Tool-augmented LLMs (Gentopia)
• Practical applications of generative models (image reconstruction, AI healthcare, etc.)

🔥 News

• 2023.11.29:  Feel free to check our new paper Zero-shot REC
• 2023.10.03:  Gentopia accepted by EMNLP 2023 system demo!
• 2023.08.05:  🎉🎉 My first personal website is established! I eagerly anticipate further research opportunities and publications will enrich its content!

📘 Preprints

arXiv preprints

Zero-shot Referring Expression Comprehension via Structural Similarity Between Images and Captions

Zeyu Han, Fangrui Zhu, Qianru Lao, Huaizu Jiang

[code (coming soon)]

• This paper presents a zero-shot referring expression comprehension method using several large foundation models, i.e., ChatGPT and Vision-Language Alignment models. We build a two-stage grounding pipeline that explicitly model the relationship between entities to refer visual objects.

📝 Publications

(^* indicates equal contribution; ^# indicates corresponding authorship.)

EMNLP 2023

Gentopia.AI: A Collaborative Platform for Tool-Augmented LLMs

Binfeng Xu, Xukun Liu, Hua Shen, Zeyu Han, Yuhan Li, Murong Yue, Zhiyuan Peng, Yuchen Liu, Ziyu Yao, Dongkuan Xu

project page

• This paper presents Gentopia, a lightweight and extensible framework for LLM-driven Agents and ALM research. It also maintains GentPool, a platform engineered to facilitate the registration and sharing of specialized agents, which seamlessly integrates GentBench, an ALM benchmark devised specifically for the comprehensive performance evaluation of agents.

MICCAI 2023

Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction (MICCAI 2023)

Zeyu Han^*, Yuhan Wang^*, Luping Zhou, Peng Wang, Binyu Yan, Jiliu Zhou, Yan Wang^#, and Dinggang Shen^#

code

• This paper presents a coarse-to-fine PET reconstruction framework using diffusion models. The coarse-to-fine design can significantly improve the overall sampling speed of our method. Furthermore, two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process, which can enhance the correspondence between the LPET image and the RPET image, further improving clinical reliability.

• Polymerized Feature-based Domain Adaptation for Cervical Cancer Dose Map Prediction, Jie Zeng^*, Zeyu Han^*, Xingchen Peng, Jianghong Xiao, Peng Wang, Yan Wang^# (ISBI 2023)

🎖 Honors and Awards

• 2022.12, 2021.12, 2020.12, National Scholarship for Encouragement, Sichuan University

📖 Educations

• 2019.09 - 2024.06, Undergraduate, Sichuan University

🐝 Academic Services

Reviewer

• International Workshop on Resource-Efficient Learning for Knowledge Discovery (KDD 2023 workshop)
• International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2023)

🐾 Miscellaneous

Piano

I enjoy playing piano in my spare time and upload some of my plays to Bilibili. Give me some likes and coins please! 😊😊

Cat

I feed a lovely cat named Crescent. She is a American Shorthair and is very cute. Her mbti is infj (same as mine). I love her very much! 🥰🥰

Some photos of her: