Abhinav M.

A new collaborator article is out on biostatistics.ca! Thanks to Renee (Hui Xin) Ng for sharing this insightful piece about using sophisticated Machine Learning (ML) algorithms to develop biological constructs related to brain aging. Specifically, the article explains why and how these models are biased, how to address these biases, and what they mean for research in this field. It's an fascinating read, check it out 👇 https://lnkd.in/ehKpzDsu

12

6 Comments

Regardless of what role you work in, there are two “musts” that everybody should be cognizant of when it comes to AI: 1. All AI/ML models produce statistical inferences, which means you can never be 100% sure the outcome will be what you expect or desire. 2. All AI/ML models are trained at a snapshot in time, meaning if an AI model works well the day after it’s trained, there’s no guarantee you’ll see continued performance down the road like a year later or so. I bring these up to highlight the importance of model validation procedures. Essentially, you want to establish a framework to make ensure you’re consistently getting the desired performance from any AI solution. And it doesn’t necessarily have to come in the form of a technical solution. It’s not ideal, but human validation can be a sufficient means of validation. Regardless of how your model validation plans manifest, you still want to ensure you have some form of plan!

26

4 Comments

🌟 Saeid Abolfazli and I are thrilled to start new batch for Applied Natural Language Processing course in collaboration with the University of Toronto School of Continuing Studies. Over 12 weeks, we’ll dive into the world of NLP and large language models (LLMs), learning how to process and analyze textual data to extract actionable insights using cutting-edge algorithms and techniques. Whether you're looking to upskill or explore NLP from scratch, this course will equip you with practical knowledge and hands-on experience. Here’s a sneak peek at what we’ll cover: 1️⃣ Week 1: Introduction to Language Data Processing 2️⃣ Week 2: Text Corpora & Preprocessing 3️⃣ Week 3: Fundamentals of Machine Learning 4️⃣ Week 4: Text Vectorization & Feature Engineering 5️⃣ Week 5: Supervised & Unsupervised Learning on Text 6️⃣ Week 6: Context-Aware Language Modeling 7️⃣ Week 7: Text Visualization & Graph Analysis 8️⃣ Week 8: Scaling with Multiprocessing & Spark 9️⃣ Week 9: Deep Learning on Text 🔟Week 10: Transformer Learning for NLP 1️⃣1️⃣ Week 11: Generative AI for NLP 1️⃣2️⃣ Week 12: Capstone Project Join us for this exciting journey into the heart of NLP! Register at http://ow.ly/drbj50B8tzp 🚀 #NLP #GenAI #UofT

39

1 Comment

The Information explores how OpenAI's reasoning models could help counteract potentially slowing progress from pretraining of larger more sophisticated models with more incremental gains over earlier models. It also notes that "relatively few developers" are using o1 models. While I haven't seen the full-fledged o1 models (and am certainly not a developer), the current models sometimes think hard about how to break code -- have seen them replace functional authentication with placeholders that you have to go back and update later, or swap out the correct variables with wrong ones -- for instance upper- or lowercasing something you didn't ask for, or adding an underscore where you didn't want one. At present, I think it still is better to have tighter feedback loops with the user -- otherwise, the genAI is thinking long and hard -- while a game of telephone can ensue where mistakes sometimes increase in the process of reflection -- about something that might not be what you wanted. A snippet from the paywalled article. The Information has done a lot of unique reporting on the subject.

4

1 Comment

🚀 Advancing Information Retrieval with APEER: Automating Prompt Engineering for LLMs! 🚀 I recently delved into a fascinating paper on APEER (Automatic Prompt Engineering Enhances LLM Reranking), a method transforming the way we approach relevance ranking in Information Retrieval (IR) using Large Language Models (LLMs). 🌟 🔍 Challenges in IR with LLMs: Current IR methods with LLMs often rely heavily on human-crafted prompts for zero-shot relevance ranking. This process is time-consuming, subjective, and lacks scalability. The complexities of integrating query and passage pairs and conducting comprehensive relevance assessments further complicate the effectiveness of existing methods. 💡 APEER: A Game-Changer APEER addresses these challenges by automating prompt engineering through iterative feedback and preference optimization. Here's how it stands out: Feedback Optimization: Continuously refine prompts based on performance metrics. Preference Optimization: Improves prompts by learning from positive and negative examples. 🌟 Key Highlights: Reduced Human Effort: APEER minimizes the need for manual prompt crafting, making the process more efficient and less reliant on human expertise. Enhanced Performance: Demonstrates significant improvements in LLM performance on IR tasks, with notable gains in metrics like nDCG@10. For example, APEER achieved an average improvement of 5.29 nDCG@10 on eight BEIR datasets over manual prompts on the LLaMA3 model. Better Transferability: Shows consistent outperformance across diverse datasets and LLM architectures, including GPT-4, LLaMA3, and Qwen2. 📊 Robust Validation: APEER has been rigorously tested on multiple datasets such as MS MARCO, TREC-DL, and BEIR, ensuring its robustness and effectiveness in various IR scenarios. 🌐 Implications: This advancement represents a significant step forward in optimizing LLM prompts for complex relevance ranking tasks. By reducing manual intervention and enhancing the efficiency of LLMs, APEER paves the way for more scalable and accurate applications in real-world IR tasks. 🛠️ Conclusion: APEER's automated approach to prompt engineering is a major leap towards more effective and scalable IR systems. It highlights the potential of integrating iterative feedback and preference optimization to refine LLM performance, offering promising avenues for future research and practical applications. Looking forward to seeing how APEER influences the landscape of Information Retrieval and beyond! #InformationRetrieval #LLM #AI #MachineLearning #NLP #DataScience #Research

1

Fully agree to Pascals Summary! (Pascal's summary quote below 👇) The results? Despite the hype around LLMs' impressive performance on medical text processing and licensing exams, they still couldn't beat tried-and-true models like SVM and XGBoost when it came to real-world clinical predictions. This held true regardless of the LLMs' size, prompting strategy, or fine-tuning. So what's going on? The researchers suggest that LLMs may have a "deficiency in clinical reasoning and decision-making" compared to purpose-built models. It's a reminder that, while LLMs are undoubtedly powerful, they're not a magic bullet for every healthcare application. As the study's authors put it: "We call for caution when practitioners adopt LLMs in clinical applications."

1

Here are some questionable research practices (QRPs) if you work on LLMs. Not exhaustive but has good coverage. "Evaluating modern ML models is hard. The strong incentive for researchers and companies to report a state-of-the-art result on some metric often leads to questionable research practices (QRPs): bad practices which fall short of outright research fraud. We describe 43 such practices which can undermine reported results, giving examples where possible. Our list emphasises the evaluation of large language models (LLMs) on public benchmarks. We also discuss "irreproducible research practices", i.e. decisions that make it difficult or impossible for other researchers to reproduce, build on or audit previous research." #ArtificialIntelligence #MachineLearning #LLM

1

Large AI models seem to be dominating the headlines. They require massive amounts of data and compute to train and even significant resources to fine-tune and run inference. Yitian Zhang et al. asked the question, "Can you get large vision model performance at ImageNet-level costs?" Their Proteus framework demonstrates how to distill a large model like DINOv2 into a smaller transformer like ViT-S using ImageNet-1K. The fine-tuning objectives focus on tokens, patches, and features, using DINOv2 as a teacher. Experiments demonstrated that Proteus matches the performance of DINOv2-g/14 and outperforms other vision foundation models across 15 benchmarks. This means that a much more efficient model can be used for inference. paper: https://lnkd.in/eRqyq5SZ code: https://lnkd.in/eQVymXGe #MachineLearning #DeepLearning #ComputerVision #DataScience __________________ Enjoyed this post? Like 👍, comment 💬, or re-post 🔄 to share with others. Click "View my newsletter" under my name ⬆️ to join 1500+ readers.

41

Excited to share our latest publication in Computers in Biology and Medicine. In collaboration with the BC Children's Hospital Research Institute, this study introduces BRI-Net, an interpretable deep-learning model that leverages adversarial training to learn an unbiased representation of data, helping identify genetic markers associated with specific phenotypes. Using genomic data from a cohort of 362 pediatric cancer patients treated with cisplatin, recruited by the Canadian Pharmacogenomics Network for Drug Safety, our model identified two statistically significant SNPs associated with cisplatin-induced ototoxicity: CERS6 rs13022792 and TLR4 rs10759932. To explore our methodology and findings in detail, check out the full paper at https://lnkd.in/gEn4fZb9. A huge thanks to my amazing co-authors and collaborators, especially Bahareh Kashani, for making this possible!

66

4 Comments

Synthetic Data for LLM applications Synthetic healthcare datasets have been around for some time. Whenever we discuss synthetic data, we often find ourselves trapped in one question: How realistic is the synthetic data? We create synthetic data because we cannot share the original data for various obvious reasons, e.g., privacy, security, etc. The challenging part is that we want the synthetic data to have all the nice characteristics of the original data. With all the challenges aside, many researchers adopted and tried to prove the value of synthetic data. This morning, I came across this well-summarized survey article, "Synthetic data in healthcare: A narrative review," by Aldren Gonzales, Guruprabha Guruswamy, and Scott Smith. According to the paper, people have used synthetic data for seven categories: 1. Simulation and Prediction Research 2. Hypothesis, Methods, and Algorithm Testing 3. Epidemiological Study/Public Health Research 4. Health IT Development and Testing 5. Education and Training 6. Public Release of Datasets 7. Linking data I have been thinking about synthetic data quite a bit these days. It's because of the Large Language Models (LLMs) like ChatGPT by OpenAI and Claude by Anthropic. I have been playing around with those LLMs. I realize these days that, even if you are an expert, it's almost impossible to predict the outputs. As we cannot logically bind the output behaviors, testing an LLM model with all possible(or available) cases is the only way. Here, I think, is the new application of synthetic data: LLM application testing. If we were to build many LLM applications, we would need a lot of synthetic healthcare data. In this case, the synthetic data do not need to be hyper-realistic. They just need to represent the "quirkiness" of the real-life use cases, so weird synthetic data should be fine. Healthcare-focused LLM applications would need to test with all sorts of available synthetic data to see if such applications produce weird outputs. We may need a mechanism to do so. I think this new use case of synthetic data will be critical in healthcare. Let's see. [1] https://lnkd.in/e8DiEH9j

64

9 Comments

Here's a very recent development that designs a neural network capable of exactly modeling semialgebraic functions--such as those found in population dynamics, kinematics, and other dynamic systems models from the social and biological sciences (see blog reference for an example). I'm excited to see how this new branch of deep learning evolves and solves complex biological and social modeling challenges: https://lnkd.in/e-3zQ4ye https://lnkd.in/e3iHpCWP Franck Kalala Mutombo, might be a good avenue for new research on more complicated epidemic models.

35

5 Comments

📢 Submitting papers to arXiv? Simplifying the process just got easier! When preparing your LaTeX code for arXiv submission, it's crucial to remove comments, unwanted figures, and clean up the code to ensure a smooth submission. To streamline this process, consider using the arxiv-latex-cleaner tool by Google Research. This tool helps you create a clean and ready-to-submit LaTeX code, making your arXiv submission hassle-free. https://lnkd.in/gwT77z2X

36

🚀 Excited to Present at NeurIPS 2024, our paper: Optimization Can Learn Johnson Lindenstrauss Embeddings 🚀 Embeddings are a cornerstone of modern data science, providing compact and meaningful representations of complex data. While randomized methods like Johnson-Lindenstrauss (JL) have long been the gold standard, they overlook the structure of real-world data. But is randomness truly necessary? Could a JL matrix be learned through optimization? In our paper with Constantine Caramanis and Christos Tzamos, we answer this question affirmatively. We propose a novel optimization-based approach inspired by diffusion models. Rather than optimizing projection matrices directly, we expand the search to the space of random solution samplers, refining them step-by-step into deterministic solutions. 📍 Join me this Thursday as I present our work. I’d love to connect with anyone curious about embeddings, optimization, or the broader implications of our research across fields. Let’s discuss how these ideas could inspire advancements in various domains. Link: https://lnkd.in/dVv6rkse #NeurIPS2024 #Optimization #MachineLearning #DataScience

58

Hello! Recently, I've been very interested in exploring graph generative models for drug discovery. I came across a paper titled “A Latent Diffusion Model for Protein Structure Generation.” This paper helped me understand how graph generative models can be used for protein structure generation tasks, the limitations of existing research, and how these issues can be addressed using generative models. I would like to share these valuable insights with you all. This paper emphasizes AI advancements in bio-medicine, particularly with generative models, but highlights the challenge of increased exploration space due to complex structures, leading to higher time and resource costs. Previous research using 3D graph autoencoders and latent 3D diffusion models improved efficiency at the molecular level but was not adequately applied to larger, complex protein generation tasks. The paper introduces a method combining an Equivariant Protein Autoencoder and Latent Diffusion. The autoencoder maps protein properties to a lower dimension while preserving these properties, using an SE(3) equivariant GNN to maintain structural characteristics. The loss functions address various elements like aug, aa, and pos. The latent diffusion model integrates geometric information with traditional DDPM to achieve SE(3) invariance and model structural information distribution. Model evaluation includes checking if generated protein backbones can fold into desired shapes, sampling efficiency, secondary structure ratios, diversity, and overall effectiveness. [Potential impact] I was personally very curious about how the latent space could be utilized in the fields of molecules and proteins. Maintaining geometric information accurately in the latent space is crucial because even a slight loss of the original characteristics can result in the generation of non-synthesizable substances. To address this issue, this paper uses an 'SE(3) equivariant GNN' and an 'invariant Latent Diffusion model,' which I believe effectively solve the problem. With this methodology, I expect that the industry could achieve fast and efficient simulations. [Materials] Paper : https://lnkd.in/g_Ue4uux Code : https://lnkd.in/g_HyC5BS Blogs : https://lnkd.in/gdS-Ww6s #LoG #LearningOnGraphs #GraphPaper

21

𝗘𝘅𝗰𝗶𝘁𝗶𝗻𝗴 𝗡𝗲𝘄𝘀 𝘄𝗶𝘁𝗵𝗶𝗻 𝘁𝗵𝗲 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 (𝗟𝗟𝗠𝘀) 𝗦𝗽𝗮𝗰𝗲! PyTorch has officially released TorchTune, a new tool designed to fine-tune Large Language Models (LLMs) with ease and efficiency. This groundbreaking tool promises to streamline the fine-tuning process, making it more accessible to researchers and practitioners alike. https://lnkd.in/eMf6dNZJ 𝗧𝗼𝗿𝗰𝗵𝘁𝘂𝗻𝗲 𝗢𝗳𝗳𝗲𝗿𝘀: ▪ 𝗣𝘆𝗧𝗼𝗿𝗰𝗵-𝗻𝗮𝘁𝗶𝘃𝗲 𝗟𝗟𝗠 𝗶𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻𝘀: Build on familiar ground with composable building blocks for popular LLMs. ️ ▪ 𝗠𝗲𝗺𝗼𝗿𝘆-𝗲𝗳𝗳𝗶𝗰𝗶𝗲𝗻𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝗿𝗲𝗰𝗶𝗽𝗲𝘀: Train on consumer-grade GPUs with 24GB VRAM – no beefy machines needed! ▪ 𝗘𝘅𝘁𝗲𝗻𝘀𝗶𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝗻𝗱 𝗲𝗮𝘀𝗲 𝗼𝗳 𝘂𝘀𝗲:  YAML configs and support for common dataset formats streamline your workflow. 𝗙𝗼𝗿 𝘁𝗵𝗼𝘀𝗲 𝗶𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗲𝗱 𝗶𝗻 𝗱𝗶𝘃𝗶𝗻𝗴 𝗱𝗲𝗲𝗽𝗲𝗿, 𝗵𝗲𝗿𝗲 𝗮𝗿𝗲 𝘀𝗼𝗺𝗲 𝗿𝗲𝘀𝗼𝘂𝗿𝗰𝗲𝘀: Code: https://lnkd.in/gEgETe5t Blog: https://lnkd.in/gP3mctA6 Tutorials: https://lnkd.in/gvh2eEsU This is a fantastic opportunity to explore and innovate, I'd be very interested in hearing your thoughts. ▪ 𝗪𝗵𝗮𝘁’𝘀 𝘆𝗼𝘂𝗿 𝘁𝗮𝗸𝗲 𝗼𝗻 𝘁𝗵𝗶𝘀?  ▪ 𝗛𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝘀𝗲𝗲 𝗧𝗼𝗿𝗰𝗵𝗧𝘂𝗻𝗲 𝗶𝗺𝗽𝗮𝗰𝘁𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗰𝘂𝗿𝗿𝗲𝗻𝘁 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗼𝗿 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵? ▪ 𝗛𝗼𝘄 𝗱𝗼 𝘆𝗼𝘂 𝘁𝗵𝗶𝗻𝗸 𝗧𝗼𝗿𝗰𝗵𝗧𝘂𝗻𝗲 𝘄𝗶𝗹𝗹 𝗶𝗺𝗽𝗮𝗰𝘁 𝘁𝗵𝗲 𝗳𝘂𝘁𝘂𝗿𝗲 𝗼𝗳 𝗺𝗮𝗰𝗵𝗶𝗻𝗲 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗮𝗻𝗱 𝗔𝗜 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗺𝗲𝗻𝘁? #AI #PyTorch #TorchTune #LLMs #MachineLearning

24

1 Comment