Anatomy of an AI Request: Where Latency and Cost Are Really Born
Dan Fu · Together AI · 35 min
Sáu stage một AI request đi qua (routing tới decode) và sáu lever giảm latency, cost: kernels, disaggregation, parallelism, routing, speculative decoding, quantization.
