ML & AI Concepts and Implementations

Google Releases Gemini Omni 1.2 Flash: 15-Second AI Video at $0.07 per Second
Concepts & ImplementationsMarkTechPost2 hours ago

Google Releases Gemini Omni 1.2 Flash: 15-Second AI Video at $0.07 per Second

Google has released Gemini Omni 1.2 Flash, the newest version of its video generation and editing model. Gemini Omni 1.2 Flash generates and edits clips up to 15 seconds natively. It also cuts the price by 30% compared with Omni 1.1 Flash. The model is generally available on Google Cloud and the Gemini API. For developers, the takeaways are longer clips, fewer re-renders and a lower bill per second.

Read original on MarkTechPost ↗
Thinking Machines Releases Inkling 2 Small, a 594B MoE With 17B Active
Concepts & ImplementationsMarkTechPost2 hours ago

Thinking Machines Releases Inkling 2 Small, a 594B MoE With 17B Active

Thinking Machines Lab has released Inkling 2 Small, the first model of its second Inkling generation. The Inkling 2 Small model is a sparse Mixture-of-Experts (MoE) with 594B total and 17B active parameters. It accepts text and images, and supports variable reasoning effort. On the Artificial Analysis Intelligence Index, it scores 31. Its predecessor, Inkling Small, scores 13 on the same index. The model is live on Tinker for fine-tuning and inference.

Read original on MarkTechPost ↗
Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster
Concepts & ImplementationsMarkTechPost2 hours ago

Zyphra ZYFA: FP8 FlashAttention for AMD MI355X, Up to 1.76x Faster

Zyphra has released ZYFA (Zyphra FlashAttention on AMD), a set of FP8 and BF16 prefill attention kernels for AMD Instinct MI355X GPUs. ZYFA is Zyphra’s FP8 attention engine for long-context prefill, and the code is open source under Apache 2.0. On real model activations, its FP8 kernels run 1.16-1.76x faster than AITER, AMD’s default kernel library. At head dimension 128, ZYFA’s FP8 error is 62-78% lower than AITER’s.

Read original on MarkTechPost ↗
Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM
Concepts & ImplementationsMarkTechPost2 hours ago

Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Buoyant, the company behind Linkerd, has released Kinference, an open source project for building inference platforms on Kubernetes. Kinference is an inference control plane for Kubernetes and vLLM. It sits in front of a fleet of vLLM workers and decides where each request goes. It applies routing policies, token rate limits, KV-cache-aware load balancing and per-tenant metrics in one layer. For dev teams serving many open-weight models to many internal clients, it targets the layer above the model server. The project is Apache 2.0 licensed on GitHub. The current release is v0.1.0-alpha.7.

Read original on MarkTechPost ↗
How to Run NVIDIA Dory 4B with llama.cpp: Loop by Loop Guide
Concepts & ImplementationsMarkTechPost2 hours ago

How to Run NVIDIA Dory 4B with llama.cpp: Loop by Loop Guide

In this tutorial, we work with Dory 1.0 4B, a small reasoning model from NVIDIA’s Nemotron Labs whose middle layers form a shared block that can be applied once, twice, three, or four times for every token. Each extra pass, or loop, spends more computation per token without adding a single parameter, so one checkpoint can run at four different compute budgets with no retraining. We build the llama.cpp branch that NVIDIA’s model card pins, download the 3.7 GB GGUF, and read the architecture straight from the file to predict what each loop should cost in parameters and cache memory. We then check those predictions against the running server, measure decode speed at every loop count, test whether the cheaper…

Read original on MarkTechPost ↗