Hugging Face and Large Language Models (LLM)

C) Copyright Elephant Scale May 10, 2023

Course Description

Today, Deep Learning can accomplish results that are nothing short of miraculous. In this course, we assume that you do not want to reinvent the wheels. Rather, you want to know what is available as low-hanging fruit. In other words, you are looking for magic but you don’t want to compete with teams who create this magic. You want to partner with them and achieve the same quality results but with a practical reasonable expense of time and resources.

After the course, you will be able to do the following tasks

  • Classifying whole sentences:
    • Getting the sentiment of a review, detecting if an email is spam, determining if a sentence is grammatically correct or whether two sentences are logically related or not
  • Classifying each word in a sentence:
    • Identifying the grammatical components of a sentence (noun, verb, adjective), or the named entities (person, location, organization)
  • Generating text content:
    • Completing a prompt with auto-generated text, filling in the blanks in a text with masked words
  • Extracting an answer from a text:
    • Given a question and a context, extracting the answer to the question based on the information provided in the context
  • Generating a new sentence from an input text:
    • Translating a text into another language, summarizing a text
  • More
    • Generating a transcript of an audio sample or a description of an image.

Course objectives

  • By the end of this course, students will know…
  • How to understand the current state of the art in Deep Learning and AI
  • How to put the claims of AI to the test
  • How to utilize the existing results through transfer learning, pre-training, and fine-tuning.
  • How to package your models for deployment.
  • How to create machine learning pipelines and improve them in production.

Audience

  • Developers, data scientists, team leads, project managers

Skill Level

  • Intermediate

Duration

  • Two days

Prerequisites

  • General familiarity with machine learning

Format

  • Lectures and hands on labs. (50% – 50%)

Lab environment

  • Zero Install: There is no need to install software on students’ machines!
  • A lab environment in the cloud will be provided for students.

Students will need the following

  • A reasonably modern laptop with unrestricted connection to the Internet. Laptops with overly restrictive VPNs or firewalls may not work properly.
    • A checklist to verify connectivity will be provided
  • Chrome browser

Detailed outline

Introduction to Deep Learning

  • Understanding Deep Learning use cases
  • Understanding AI / Machine Learning / Deep Learning
  • Data and AI
  • AI vocabulary
  • Hardware and software ecosystem
  • Understanding types of Machine Learning (Supervised / Unsupervised / Reinforcement)

Computer Vision

  • Introducing Convolutional Neural Networks (CNN)
  • CNN architecture
  • CNN concepts
  • Lab: Image recognition using CNNs

Recurrent Neural Networks

  • Introducing RNNs
  • RNN architecture
  • RNN concepts
  • LSTM (Long Short Term Memory) networks
  • LSTM architecture
  • Lab: RNNs for text and sequence prediction

HuggingFace

  • Transformers
  • Encoders
  • Decoders
  • Sequence to sequence
  • Bias and limitations
  • Pipeline
  • Models
  • Tokenizers
  • Putting it all together

Fine tuning a pretrained model

  • Processing the data
  • Fine-tuning a model with the Trainer API or Keras
  • A full training

Sharing models and tokenizers

  • The Hugging Face Hub
  • Using pretrained models
  • Sharing pretrained models

Main NLP tasks

  • Token classification
  • Fine-tuning a masked language model
  • Translation
  • Summarization
  • Training a causal language model from scratch
  • Question answering
  • Mastering NLP