GitHub

FlexAttention based, minimal vllm-style inference engine for fast Gemma 2 inference.

This project has no flash-attn dependency, no custom triton kernel. Everything is implemented with FlexAttention. The code is commented, the structure is flat. Read the accompanying write-up: vLLM ...
IEEE

Design Considerations for LLM Inference in Data Centers: Chip and Interconnect

Abstract: This paper presents a cost-efficient chip prototype optimized for large language model (LLM) inference. We identify four key specifications – computational FLOPs (flops), memory bandwidth ...
IEEE

Causal Inference-Based Fault Diagnosis and Abnormal Degradation Detection for Aero-Engine

Abstract: Aero-engine fault diagnosis faces challenges such as low accuracy and weak physical interpretability. Additionally, early anomalies are difficult to identify due to complex thermodynamic ...
SDxCentral

AI inferencing will define 2026, and the market's wide open

“I get asked all the time what I think about training versus inference – I'm telling you all to stop talking about training versus inference.” So declared OpenAI VP Peter Hoeschele at Oracle’s AI ...
GitHub

Elivis-AI/vllm_musa_autodl

非常感谢踩坑文章中国科学院大学GPU架构与编程大作业二 摩尔线程赛道 (MTT S4000) AUTODL部署与测试指南 - 求索者freedom的文章 ...