About Me

I am currently a third-year Ph.D. student at NExT++ Research Center, advised by Prof. Tat-Seng Chua in School of Computing at National University of Singapore, after I obtained my M.S. and Bachelor degrees from Wuhan University.

My research revolves around Multimodal Understanding and Multimodal Generation, with a major focus on Multimodal Large Language Models and Scene Graph Modeling. Also, certain part of my intearest has been paid on Natural Language Processing.

Some of my representive work:
NExT-GPT: The first unified any-to-any multimodal LLM, capable of understanding and generating across any modality or combination of modalities (e.g., text, image, video, audio).
[PDF] [Github] [Huggingface] [Video]
(ICML'24 Oral, selected as a Most Influential Paper by Paper Digest, WAIC Youth Outstanding Paper Award, , )
Any2Caption: A SoTA framework for controllable video generation from any condition by being the first to leverage MLLMs to interpret diverse inputs into dense, structured captions.
[PDF] [Github] [Huggingface] [Video]
(Preprint, 2025)
Setok: The first to propose a general dynamic semantic-equivalent vision tokenizer, fundamentally enhancing the performance bottlenecks of existing MLLMs.
[PDF] [Github]
(ICLR'25)
USG: The first to propose a Universal Scene Graph representation framework that unifies structured semantic scene graphs across modalities including images, text, videos, and 3D.
[PDF] [Github]
(CVPR'25 Highlight)

πŸ”₯NEWsπŸ”₯

  • [NEW!-2025/03] We release the first survey on MM-CoT reasoning, welcom to participate and star✨✨.
  • [NEW!-2025/03] 🐾 I will be a volunteer at ICLR 2025. One paper (Setok) is accpted by ICLR-25. Looking forward to meet you.
  • [NEW!-2024/02] πŸ₯³πŸ₯³πŸ₯³ My two full papers are accpted in CVPR 2025: Universal Scene Graph Generation and 4D Scene Graph Generation .
  • [NEW!-2025/02] Welcome to SSNLP-25, Free Registration!!!
  • [NEW!-2024/12] πŸ₯³ I maintain a github repo focusing on Awesome-Scene-Graph-Generation-and-Application. Welcome to check it out, contribute, and share any resources or insights related to scene graph generation!
  • [NEW!-2024/12] One paper focusing on mitigating hallucination in MLLMs is accepted by AAAI-25!
  • [NEW!-2024/08] 🐾I’m excited to have the opportunity to be a volunteer at ACL 2024. Looking forward to being part of it!
  • [NEW!-2024/08] I have tried to build NExT-GPT in PaddlePaddle, PaddleNLP and PPDiffusers. Still working on it. πŸ€”πŸ€”
  • [NEW!-2024/05] Congratulations! πŸ₯³πŸ₯³πŸ₯³, two full papers are accpted in ICML-24, NExT-GPT and Video-of-Thought .
  • [NEW!-2024/04] We release Vitron (Demo, Paper , Code), a universal pixel-level vision LLM designed for understanding, generating, segmenting, editing of both image and video. 🌟🌟
  • [NEW!-2024/02] One full paper is accepted by CVPR-24 about Text-to-Video Generation. Congrats to all my co-authors.
  • [NEW!-2024/01] I honor Baidu Scholarship (10 people worldwide). πŸ₯³πŸ₯³πŸ₯³
  • [NEW!-2023/12] I have successfully passed my Rearcsh-based QE, I am now a Ph.D. Candidate. πŸ₯³πŸ₯³πŸ₯³
  • [NEW!-2023/11] I'll join Kunlun 2050 Research as a research intern, advised by Prof. Yan. πŸ˜†
  • [NEW!-2023/10] We build NExT-GPT, a general-purpose any-to-any MLLM. 🌟
  • [NEW!-2023/09] One full paper is accepted in NeurIPS-23, about Intricate Text-to-image Generation based on Scene Graph.
  • [NEW!-2023/07] One full paper is accepted in ACM MM-23, about High-faithful Text-to-image Generation enhanced with layout planning from LLM.
  • [NEW!-2023/05] Two full papers is accepted in ACL-23, about Multimodal Relation Extraction and Image Captioning.
  • [NEW!-2022/07] I'm heading to SoC NUS to pursue my PhDβ€”a new journey and fresh challenges await! πŸ™ƒ
    		•

    Publications

    2025
    • Yaoting Wang, Shengqiong Wu, Yuechen Zhang, William Wang, Ziwei Liu, Jiebo Luo, Hao Fei.  Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey. arxiv. 2025.  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Tat-Seng Chua, Shuicheng Yan.  Universal Scene Graph Generation. CVPR. 2025.  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Jingkang Yang, Xiangtai Li, Juncheng Li, Hanwang Zhang, Tat-seng Chua.  Learning 4D Panoptic Scene Graph Generation from Rich 2D Visual Scene. CVPR. 2025.  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Xiangtai Li, Jiayi Ji, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan.  Towards Semantic Equivalence of Tokenization in Multimodal LLM. ICLR. 2025.  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Liangming Pan, William Yang Wang, Shuicheng Yan, Tat-Seng Chua.  Combating Multimodal LLM Hallucination via Bottom-up Holistic Reasoning. In Proceedings of AAAI. 2025.   [pdf]
    2024
    • Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan.  VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing. In Proceedings of NeurIPS. 2024.  [Code][pdf]
    • Meng Luo, Hao Fei*, Bobo Li, Shengqiong Wu, Qian Liu, Soujanya Poria, Erik Cambria, Mong-Li Lee, Wynne Hsu.  PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis. In Proceedings of ACM MM. 2024.   (Oral).[Code][pdf]
    • Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, Tat-Seng Chua.  NExT-GPT: Any-to-Any Multimodal Large Language Model. In Proceedings of ICML. 2024.   (Oral) [Code | 3.5k 🌟][pdf]
    • Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, Wynne Hsu.  Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition. In Proceedings of ICML. 2024.   (Oral) [Code][pdf]
    • Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Tat-Seng Chua.  Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs. In Proceedings of CVPR. 2024.  [Code][pdf]
    2023
    • Shengqiong Wu, Hao Fei, Hanwang Zhang, Tat-Seng Chua.  Imagine That! Abstract-to-Intricate Text-to-Image Synthesis with Scene Graph Hallucination Diffusion. In Proceedings of NeurIPS. 2023.  (long, poster)  [Code][pdf]
    • Leigang Qu*, Shengqiong Wu*, Hao Fei, Liqiang Nie, Tat-Seng Chua.  LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation. In Proceedings of ACM MM. 2023.  (*: equal contribution, long)  [Code][pdf]
    • Bobo Li, Hao Fei, Yuhan Wu, Jinsong Zhang, Shengqiong Wu, Jingye Li, Yijiang Liu, Lizi Liao, Tat-Seng Chua, Fei Li, Donghong Ji.  DiaASQ: A benchmark of conversational aspect-based sentiment quadruple analysis.In Proceedings of ACL. 2023.  (long, poster)  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Yixin Cao, Lidong Bing, Tat-Seng Chua.  Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling. In Proceedings of ACL. 2023.  (long, poster, paper award nomination, 1.6%)  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Wei Ji, Tat-Seng Chua.  Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment. In Proceedings of ACL. 2023.  (long, oral)  [pdf]
    2022
    • Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, Tat-Seng Chua.  LasUIE: Unifying information extraction with latent adaptive structure-aware generative language model. In Proceedings of NeurIPS. 2022.  (long, poster) [Code][pdf]
    • Hu Cao, Jingye Li, Fangfang Su, Fei Li, Hao Fei, Shengqiong Wu, Bobo Li, Liang Zhao and Donghong Ji.  OneEE: A One-Stage Framework for Fast Overlapping and Nested Event Extraction. In Proceedings of COLING. 2022.  (long, oral) [Code][pdf]
    • Shengqiong Wu, Hao Fei, Fei Li, Meishan Zhang, Yijiang Liu, Chong Teng, Donghong Ji.  Mastering the Explicit Opinion-Role Interaction: Syntax-Aided Neural Transition System for Unified Opinion Role Labeling. In Proceedings of AAAI. 2022.  (long, online) [Code][pdf]
    • Jingye Li, Hao Fei, Jiang Liu, Shengqiong Wu, Meishan Zhang, Chong Teng, Donghong Ji, Fei Li.  Unified named entity recognition as word-word relation classification. In Proceedings of AAAI. 2022.  (long, online) [Code][pdf]
    2021
    • Shengqiong Wu, Hao Fei, Yafeng Ren, Donghong Ji, Jingye Li.  Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge. In Proceedings of IJCAI. 2021.  (long, online) [Code][pdf]

    Academic Services

    Conference Reviewer
    NeurIPS-23/24, ICLR-24/25, ICML-24/25, CVPR-24/25, ACM MM-23/24, IJCAI-23/24, AAAI-24/25, ACL-23/24, WSDM-23,
    Journal Reviewer
    TOMM, IEEE/ACM TALLIP, IPM, KBS, Neurocomputing

    Invited Talks

    1. [2025/03]       Towards Semantic Equivalence of Tokenization in Multimodal LLM. NICE.
    2. [2024/01]       The Path to AGI: Achieving Modality Unification with NExT-GPT. Qingyuan Talk.
    3. [2023/12]       NExT-GPT: Any-to-Any Multimodal LLM. AI New Youth.
    4. [2022/12]       Deep Learning based Natural Language Processing: A Survey and Outlook. Jianghan University, China.
    5. [2021/10]       Comparison of Aspect-based Sentiment Analysis based on span model and transition model. Wuhan University, China.
    6. [2021/05]       Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge. CIPS, Youth Working Committee.

    Internships


    Dec. 2024 - Now                    Kauishou, Remote
                                                              VGI Research Intern
                                                              Advisor: Weicai Ye, Xintao Wang

    Nov. 2023 - Jun. 2024         Kunlun Skywork AI, Singapore
                                                              2050 Research Intern
                                                              Advisor: Shuicheng Yan, Director

    Jul. 2019 - Aug. 2019          China Merchants Bank, Wuhan, China
                                                              Information Technology Department Intern
                                                              Advisor: Hua Pan, General Manager

    Jul. 2018 - Nov. 2018          YITU, ShangHai, China
                                                              Solution Engineer Intern
                                                              Advisor: Ze Deng

    Honors & Awards

  • [2025]    Dean's Graduate Research Excellence Award. NUS.
  • [2024]    Google PhD Fellowship. Google.
  • [2024]    ByteDance Scholarship, ByteDance.
  • [2024]    Baidu Scholarship, Baidu.
  • [2023]    Research Achievement Award of SoC, NUS.
  • [2022]    President's Graduate Fellowship of NUS.
  • [2022]    Award of Excellent Master Thesis of Hubei Cyberspace Security Society.
  • [2022]    Excellent Graduated Graduate Student.
  • [2021]    Huawei Scholarship for graduate students.
  • [2021]    National Scholarship for graduate students.
  • [2021]    1st prize of Excellent Scholarship of Academy for M.S. Fellowship.
  • [2021]    1st prize of Extraordinary scholarship of academy.
  • [2020]    Outstanding graduate student award.
  • [2020]    2st prize of academic scholarship of Wuhan University for graduate.
  • [2017]    Excellent student cadre.
  • [2016-2019]    2st prize of academic scholarship of Wuhan University for undergraduate.
  • [2017]    National Encouragement Scholarship.
    		•

    Education


    Aug. 2022 - Now                        School of Computing,   National University of Singapore,   Singapore
                                                                  Ph.D. in Computer Science

    Sep. 2019 - Jun. 2022             School of Cyber Science and Engineering,   Wuhan University,   China
                                                                  M.S. in Computer Science

    Sep. 2015 - Jun. 2019             School of Computer Science,   Wuhan University,   China
                                                                  B.S. in Computer Science and Technology

    Skills and MISC

  • Coding
                                       ...
  • Deep Learning
         ...
  • Hobby
         ...