With the growing model size of deep neural networks (DNN), deep learning training is increasingly relying on handcrafted search spaces to find efficient parallelization execution plans. However, our ...

What if the key to unlocking faster, more efficient AI development wasn’t just in the algorithms you write, but in the hardware you choose? For years, the debate between Google’s Tensor Processing ...

The rise of large language models (LLMs) has transformed natural language processing, but training these models comes with significant challenges. Training state-of-the-art models like GPT and Llama ...

Imagine a world where the wait for your 3D-printed masterpiece shrinks from a day and a half to just a few hours. It sounds like a dream, right? If you’ve ever felt the frustration of watching your ...

A repository to store all of my epidemiological predictive models, including a basic SIR model and it's modifications. This repository also has some other fun models, including the SIInZD model for ...

Monad Labs raised $225 million, led by Paradigm, pushing forward the discussion on parallelized EVM chains. Monad is a new layer-1 smart contract platform that recently raised $225 million in funding ...

A team of researchers in Japan released Fugaku-LLM, a large language model [1] with enhanced Japanese language capability, using the RIKEN supercomputer Fugaku. The team is led by Professor Rio Yokota ...

PyTorch introduced TK-GEMM, an optimized Triton FP8 GEMM kernel, to address the challenge of accelerating FP8 inference for large language models (LLMs) like Llama3 using Triton Kernels. Standard ...

Abstract: This paper discusses a novel method for parallelization of a Shift and Add Reducer. This improvement allows for high levels of parallelization and decreases the number of cycles required for ...