llms

class mfai.pytorch.models.llms.FreezeMLMMixin

Bases: object

A Mixin for (un)freezing llm and vision stages of a multimodal model.

backend: GPT2 | Llama2 | CrossAttentionGPT2 | Llama3

freeze_llm()

Freeze the LLM layers (not the vision layers).

Return type:

None

freeze_vision()

Freeze the vision encoder layers.

Return type:

None

unfreeze_llm()

Unfreeze the LLM layers.

Return type:

None

unfreeze_vision()

Unfreeze the vision encoder layers.

Return type:

None

vision_encoder: Module

cross_attention
fuyu
gpt2	Pytorch implementation of GPT-2.
llama2	Pytorch implementation of Llama2.
llama3	Llama3 standalone implementation inspired from https://github.com/rasbt/LLMs-from-scratch/blob/main/ch05/07_gpt_to_llama/standalone-llama32.ipynb Explanations on grouped query attention: https://www.ibm.com/think/topics/grouped-query-attention.