School/Faculty: Computer Science
Eligibility: UK/International
Funding: School of Computer Science Studentship consisting of the award of fees, together with a tax-free maintenance grant of £20,780 per year for 3.5 years.
Lead Supervisor’s full name & email address
Dr Massimiliano Fasi: m.fasi@leeds.ac.uk
Project summary
The growing importance of artificial intelligence is fostering a paradigm shift in the world of hardware design. On the one hand, the increasing complexity of deep-learning models demands computers faster and more powerful than ever before. On the other hand, the numerical formats available in conventional hardware are often too accurate for the needs of machine learning: they do not improve the quality of the trained model but may deteriorate it by causing overfitting of the training data.
Hardware vendors have begun to design specialised hardware accelerators that can perform very efficiently a limited range of operations using low-precision formats such as FP8, binary16, BFLOAT16, TensorFloat-32, and a number of other custom fewer-than-32-bit formats. At present, there is no widely-adopted standard that systematises low-precision computer arithmetic, and two hardware vendors may decide to implement a same format slightly differently, leading to irreproducible results, non-portable code, hard-to-find bugs, and other unexpected behaviours.
On the backdrop of this complex hardware landscape, this PhD project can explore one or more of several directions.
- The exact design of commercial hardware is a jealously guarded secret, and the numerical behaviour of a specific device has to be probed by a combination of theoretical analysis and reverse engineering, in particular for operations that involve inputs and outputs in different formats. Currently, this probing must be done manually and on a case by case basis – a process that is long, tedious, and prone to errors. We can streamline this process, to make it more accessible to users without deep floating-point knowledge.
- One can leverage low-precision accelerators for scientific computing by using a number of tricks, known as "mixed-precision" algorithms. Developing such algorithms is far from trivial. We can look at computational kernels that are common in scientific workloads and rewrite them to achieve better performance while delivering high accuracy despite running on low-precision hardware.
- Hardware is designed to cater for specific applications, but this is only effective if the requirements of the application are fully specified. By means of a theoretical and experimental study, we can better understand the specific needs of different applications, and we can use this knowledge to inform the development of tomorrow's hardware. A typical outcome of this line of work is the design of ISA extensions, and a particularly suitable starting point for this is the RISC-V ISA.
Please state your entry requirements plus any necessary or desired background
A first class or an upper second class British Bachelors Honours degree (or equivalent) in an appropriate discipline.
Subject Area
Computer Science & IT, Electrical & Electronic
Keywords
Artificial intelligence, floating-point arithmetic, numerical analysis, computer arithmetic, machine learning
£20,780 per year for 3.5 years
