resnet

class mfai.pytorch.models.resnet.ResNet50(num_channels=3, num_classes=1000, input_shape=None, settings=ResNet50Settings(encoder_depth=5, encoder_weights=False, encoder_stride=32))

Bases: Module

Parameters:

• num_channels (int)

• num_classes (int)

• input_shape (tuple[int, int] | None)

• settings (ResNet50Settings)

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses. :rtype: Tensor

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

x (Tensor)

Return type:

Tensor

settings_kls

alias of ResNet50Settings

class mfai.pytorch.models.resnet.ResNet50MLM(num_channels=3, num_classes=1000, input_shape=None, settings=ResNet50MLMSettings(encoder_depth=5, encoder_weights=False, encoder_stride=32, num_tokens=32, pos_embedding=False, mlp_output=False))

Bases: Module

A ResNet50 model adapted for Multi-Modal Language Models (MLM). This model outputs a sequence of feature vectors instead of a single classification output.

Parameters:

• num_channels (int)

• num_classes (int)

• input_shape (tuple[int, int])

• settings (ResNet50MLMSettings)

forward(x)

Forward function of the ResNetMLM vision encoder.

Parameters:

x (Tensor) – tensor of shape (B, num_channels, height, width)

Returns:

tensor of shape (B, num_tokens, num_classes)

Return type:

Tensor

settings_kls

alias of ResNet50MLMSettings

class mfai.pytorch.models.resnet.ResNet50MLMSettings(encoder_depth=5, encoder_weights=False, encoder_stride=32, num_tokens=32, pos_embedding=False, mlp_output=False)

Bases: object

Parameters:

• encoder_depth (int)

• encoder_weights (bool)

• encoder_stride (int)

• num_tokens (int)

• pos_embedding (bool)

• mlp_output (bool)

encoder_depth: int

encoder_stride: int

encoder_weights: bool

classmethod from_dict(kvs, *, infer_missing=False)

Return type:

TypeVar(A, bound= DataClassJsonMixin)

Parameters:

kvs (dict | list | str | int | float | bool | None)

classmethod from_json(s, *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw)

Return type:

TypeVar(A, bound= DataClassJsonMixin)

Parameters:

s (str | bytes | bytearray)

mlp_output: bool

num_tokens: int

pos_embedding: bool

classmethod schema(*, infer_missing=False, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Return type:

SchemaF[TypeVar(A, bound= DataClassJsonMixin)]

Parameters:

• infer_missing (bool)

• many (bool)

• partial (bool)

to_dict(encode_json=False)

Return type:

Dict[str, Union[dict, list, str, int, float, bool, None]]

to_json(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, indent=None, separators=None, default=None, sort_keys=False, **kw)

Return type:

str

Parameters:

• skipkeys (bool)

• ensure_ascii (bool)

• check_circular (bool)

• allow_nan (bool)

• indent (int | str | None)

• separators (Tuple[str, str] | None)

• default (Callable | None)

• sort_keys (bool)

class mfai.pytorch.models.resnet.ResNet50Settings(encoder_depth=5, encoder_weights=False, encoder_stride=32)

Bases: object

Parameters:

• encoder_depth (int)

• encoder_weights (bool)

• encoder_stride (int)

encoder_depth: int = 5

encoder_stride: int = 32

encoder_weights: bool = False

classmethod from_dict(kvs, *, infer_missing=False)

Return type:

TypeVar(A, bound= DataClassJsonMixin)

Parameters:

kvs (dict | list | str | int | float | bool | None)

classmethod from_json(s, *, parse_float=None, parse_int=None, parse_constant=None, infer_missing=False, **kw)

Return type:

TypeVar(A, bound= DataClassJsonMixin)

Parameters:

s (str | bytes | bytearray)

classmethod schema(*, infer_missing=False, only=None, exclude=(), many=False, context=None, load_only=(), dump_only=(), partial=False, unknown=None)

Return type:

SchemaF[TypeVar(A, bound= DataClassJsonMixin)]

Parameters:

• infer_missing (bool)

• many (bool)

• partial (bool)

to_dict(encode_json=False)

Return type:

Dict[str, Union[dict, list, str, int, float, bool, None]]

to_json(*, skipkeys=False, ensure_ascii=True, check_circular=True, allow_nan=True, indent=None, separators=None, default=None, sort_keys=False, **kw)

Return type:

str

Parameters:

• skipkeys (bool)

• ensure_ascii (bool)

• check_circular (bool)

• allow_nan (bool)

• indent (int | str | None)

• separators (Tuple[str, str] | None)

• default (Callable | None)

• sort_keys (bool)

class mfai.pytorch.models.resnet.ResNetEncoder(out_channels, depth=5, **kwargs)

Bases: ResNet

Resnet with encoder functionality such as: - output channels specification of feature tensors (produced by encoder). - patching first convolution for arbitrary input channels.

Parameters:

• out_channels (tuple[int, ...])

• depth (int)

• kwargs (Any)

forward(x)

Define the computation performed at every call.

Should be overridden by all subclasses. :rtype: list[Tensor]

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

Parameters:

x (Tensor)

Return type:

list[Tensor]

load_state_dict(state_dict, **kwargs)

Copy parameters and buffers from state_dict into this module and its descendants.

If strict is True, then the keys of state_dict must exactly match the keys returned by this module’s state_dict() function.

Warning

If assign is True the optimizer must be created after the call to load_state_dict unless get_swap_module_params_on_conversion() is True.

Parameters:

• state_dict (dict) – a dict containing parameters and persistent buffers.

• strict (bool, optional) – whether to strictly enforce that the keys in state_dict match the keys returned by this module’s state_dict() function. Default: True

• assign (bool, optional) – When set to False, the properties of the tensors in the current module are preserved whereas setting it to True preserves properties of the Tensors in the state dict. The only exception is the requires_grad field of Parameter for which the value from the module is preserved. Default: False

• kwargs (Any)

Returns:

• missing_keys is a list of str containing any keys that are expected

  by this module but missing from the provided state_dict.

• unexpected_keys is a list of str containing the keys that are not

  expected by this module but present in the provided state_dict.

Return type:

NamedTuple with missing_keys and unexpected_keys fields

Note

If a parameter or buffer is registered as None and its corresponding key exists in state_dict, load_state_dict() will raise a RuntimeError.

make_dilated(output_stride)

Return type:

None

Parameters:

output_stride (int)

property out_channels: tuple[int, ...]

Return channels dimensions for each tensor of forward output of encoder.

property output_stride: int

set_in_channels(in_channels, pretrained=True)

Change first convolution channels.

Return type:

None

Parameters:

• in_channels (int)

• pretrained (bool)

mfai.pytorch.models.resnet.get_resnet_encoder(name, in_channels=3, depth=5, weights=True, output_stride=32)

Return an encoder with pretrained weights or not.

Return type:

ResNetEncoder

Parameters:

• name (Literal['resnet18', 'resnet34', 'resnet50'])

• in_channels (int)

• depth (int)

• weights (bool)

• output_stride (int)