Patches often include configurations to leverage ALiBi (Attention with Linear Biases) for handling longer sequences (e.g., 65k tokens) on consumer hardware.
# Broadcast to query_len mask = mask.expand(batch, 1, query_length, key_length) patch mpt
MPT (Modified Transformer) uses ALiBi or Rotary embeddings. This patch fixes rotary position cache invalidation and attention mask expansion for variable-length sequences in a custom MPT block. patch mpt
Known to drop executable files into temporary directories and write unauthorized API strings. patch mpt