Back Cover, Volume 43, Issue 7
Human mutation（2022）
Programa de Pós‐Graduação em Ciências da Saúde Universidade de Brasília Brasília DF Brasil | Programa de Pós‐Graduação em Ciências Médicas Universidade de Brasília Brasília DF Brasil | Department of Molecular and Human Genetics Baylor College of Medicine Houston Texas USA | Department of Pediatrics University of Washington Seattle Washington USA | Programa de Pós‐graduação em Ciências Genômicas e Biotecnologia | Radboud University Nijmegen Medical Centre Nijmegen The Netherlands | Faculdade de Medicina Universidade de Brasília Brasília DF Brasil | Medical Genetics Department | Division of Genetics and Genomic Medicine | Obstetrics and Gynecology Stanford University Stanford California USA | Unidad de Genética | Department of Orthopedics Indira Gandhi Medical College Snowdon India | Hospital de Reabilitação de Anomalias Craniofaciais Bauru Brasil | Universidade Presbiteriana Mackenzie–UPM São Paulo Brasil | Laboratório de Endocrinologia Celular e Molecular LIM25 | Unidade de Genética | Programa de Mestrado Profissional em Aconselhamento Genético e Genômica Humana | Department of Pathology and Laboratory Medicine | Department of Medical Genetics Basaksehir Cam and Sakura City Hospital Istanbul Turkey | School of Medicine Istanbul Medeniyet University Istanbul Turkey | Medical Genetics Department Medicine Faculty Mugla Sitki Kocman University Mugla Turkey | Department of Medical Genetics Osaka Women's and Children's Hospital Osaka Japan | Saitama Children's Medical Center | Human Genome Sequencing Center | Departamento de Genética e Biologia Evolutiva | Department of Genetics and Genomics
- Pretraining has recently greatly promoted the development of natural language processing (NLP)
- We show that M6 outperforms the baselines in multimodal downstream tasks, and the large M6 with 10 parameters can reach a better performance
- We propose a method called M6 that is able to process information of multiple modalities and perform both single-modal and cross-modal understanding and generation
- The model is scaled to large model with 10 billion parameters with sophisticated deployment, and the 10 -parameter M6-large is the largest pretrained model in Chinese
- Experimental results show that our proposed M6 outperforms the baseline in a number of downstream tasks concerning both single modality and multiple modalities We will continue the pretraining of extremely large models by increasing data to explore the limit of its performance
