Deep Learning and LLM

Copyright Elephant Scale May 16, 2023

Course Description

  • This course concentrates on open-source models other than ChatGPT.
  • Large Language Models (LLM) are taking the world by storm.
  • HuggingFace provides libraries and a place to put LLMs in production.
  • This course introduces the students to AI, Neural Networks, and LLMs.

After the course, you will be able to do the following tasks

  • Evaluate LLM models
  • Put the models into production
  • Use HuggingFace as a possible implementation platform

Course objectives

  • By the end of this course, students will know…
  • How to understand the current state of the art in Deep Learning and AI
  • How to put the claims of AI to the test
  • How to utilize the existing results through transfer learning, pre-training, and fine-tuning.
  • How to package your models for deployment.
  • How to create machine learning pipelines and improve them in production.

Audience

  • Developers, data scientists, team leads, project managers

Skill Level

  • Intermediate

Duration

  • Two days

Prerequisites

  • General familiarity with machine learning

Format

  • Lectures and hands-on labs. (50% – 50%)

Lab environment

  • Zero Install: There is no need to install software on students’ machines!
  • A lab environment in the cloud will be provided for students.

Students will need the following

  • A reasonably modern laptop with unrestricted connection to the Internet. Laptops with overly restrictive VPNs or firewalls may not work properly.
    • A checklist to verify connectivity will be provided
  • Chrome browser

Detailed outline

Introduction to Deep Learning

  • Understanding Deep Learning Use Cases
  • Overview of Neural Networks: NN, CNN, RNN.

Prompt engineering for LLMs

  • Best practices
  • Practical advice

HuggingFace offering

  • Transformers library
  • Models
  • Putting it all together

Fine-tuning a pre-trained model

  • Processing the data
  • Fine-tuning a model with the Trainer API or Keras
  • A full training

Main NLP tasks

  • Token classification
  • Fine-tuning a masked language model
  • Translation
  • Summarization
  • Training a causal language model from scratch
  • Question answering
  • Mastering NLP

Open LLM

  • Overview of LLMs available
  • Comparison of capabilities
  • Evaluating and fine-tuning an LLM
  • Alpaca from Stanford
  • LLama from Facebook
  • Dolly from Databricks
  • Nomic
  • Vicuna