PyTorch: a reference language
A reference implementation is a simplified but complete version of a system that trades performance in return for clarity. We might then say a reference “language” is the fabric of APIs and conventions from which these implementations are cut. At first glance, PyTorch obviously is a reference language: it is, after all, commonly called the lingua franca of modern deep learning. But upon a closer look, there is confusion: Reference implementations usually aren’t deployed to production. But I do my training jobs with PyTorch! Everyone’s writing kernels with kernel DSLs. What is the role of PyTorch if it’s just gluing kernels together? AI coding will eventually mean that any stack can be …
Continue reading →