Technology Readiness Levels – are they fit for purpose?

I am often asked to describe “Technology Readiness Levels” and their application, and to justify their use.

What are they?

Several approaches have been developed to measure the maturity or readiness of a technology, system (component) or product under development. From these, Technology Readiness Levels (TRLs) have are used widely. TRLs originated in NASA, and consist of nine discrete levels of maturity of a single technology. The definitions of each level are sometimes modified by organisations to tailor them to their own specific needs. There are often differences in the way TRLs are applied.

An obvious question is whether the terms ‘Technology Maturity’ and ‘Technology Readiness’ mean the same thing?  In my view, although they are related, they are definitely not synonymous. Technology readiness (level) is only one dimension of technology maturity; other indicators include how widely available it is, how many applications it has been used in (and with what success), etc. They help you gain a summary indication of the maturity of a technology, but they are not definitive in
and of themselves. For example, system readiness, which is very application / context
specific, is also an important factor in assessing maturity. Provided their limitations are understood, they are a useful guide. They do not provide a complete view of the maturity of a technology.

Are they useful?

The advantages of TRLs include:

  • Quite widely undertstood and used,
  • Simple to articulate and explain, and
  • They cover the scale of development reasonably well.

Disadvantages:

  • Too coarse,
  • Too difficult to measure / assess in a consistent way,
  • Too insensitive to application and context.

I have used TRLs as an aid to help judge the progress in R&D projects, to set targets and timescales for R&D activities, to select between candidate technologies, and to help position technology developments within product development plans.

How do you use them?

Essentially, by considering criteria for the transition between TRLs. In practice, I find that this often boils down to looking closely at the transitions between TRL 3 and 4, and between TRL 6 & 7. The former can be thought of as the boundary between science and engineering, so the criteria are around assessing the evidence that the science is understood well enough to enable engineering to start. The latter is the point at which there is a transition from (conceptual) prototype to actual product.

Are there any alternatives to using TRLs?

System Readiness Levels (SRLs) and Exploitation Readiness Levels are also relevant, but it always has to be remembered that these are just other ways of summarising the accumulation of evidence of maturity and not a substitute for it.

First of all, SRLs aren’t more useful than TRLs: neither is sufficient by themselves. As to determining SRLs, essentially this boils down to being able to demonstrate that system level behaviour corresponds to the predictions made from models of system components (technologies) and their interactions with each other and the external operating environment. The goal is to achieve a sufficient level of fidelity in this correspondence to provide evidence that the knowledge you have of the system is sufficiently reliable to risk releasing the system into the wild. A whole range of techniques might be used to do this, from analysis, modelling and simulation through to design reviews, HWIL tests, and field trials, for example.

Are the TRL definitions adequate?

The language used in the most common definitions, such as those referenced above, is largely application domain agnostic and reasonably clear, though some terms and expressions are vague and ambiguous. It is also “unscientific” in the sense that it implies progress is made by proving (validating) your predictions rather than by attempting to falsify them (see Karl Popper - more on this later). As a result it could be seen as encouraging a very self referencing approach to development in which solutions are tuned to specific cases and wider requirements, especially non functional ones such as performance requirements are neglected. When this happens – and in my experience this is far too often the case – the solution is really a pencil balanced on its point.

Also, the 9 TRL levels are probably neither sufficient nor appropriate for assessing all technologies. Although they cover the ground from a first inkling of an idea to actually using it in anger, they don’t necessarily distinguish very well between approaches sometimes seen in the software world where the product is either made bullet proof before it is released, or the vendor relies on its customers to debug it, i.e., release occurs at different levels of technology maturity. It also seems to imply the existence of uncharted territory below TRL 1 but doesn’t really suggest a sound reason as to why it is excluded.

What does TRL9 mean?

Ideally it means it is ready to be used in an application that predictably meets the requirements against which it has been designed. But TRLs as such are too narrow a view of all the things that feed into this readiness (maturity) not least of which is the context.

Are there enough TRLs?

Probably not - most simply, TRL 0 could articulate the properties of scientific knowledge in its least mature forms, while TRL 10 could do the same for those technologies that have matured to the state of commodities, i.e., where technical knowledge is so complete and reliable that all the design engineer has to do is look it up.

Are TRLs “agile”?

The main implication for TRLs of changing requirements during development is that a technology previously believed to be at TRL6 might have to be reassessed as being at TRL4, and if this is done rigorously it can provide extremely useful insight into what it is you then need to be agile about.

Time and cost of the development are typically so specific to the technology and the wider context in which it is being developed and applied that they absolutely should be estimated and represented but alongside TRLs not as part of them. Also, they aren’t independent of each other – to some extent engineering is about trading time into cost!

 

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>