Ted Metcalfe provides me (us) with much insight and deep thinking about engineering, competency, ethical behaviour, learning from (engineering/technical) failures, and all kinds of other issues and aspects of life. He’s semi-retired but thankfully not sailing off into the sunset yet.

He often talks about raising the competency/skills of up-and-coming engineers.

We both share that interest, and both want to be there to help with that, but we sort of disagree on whether, while raising that competency, we should be testing people after or along the way.

Like, I think he means, bona fide, marked, challenging, no cheating, sweat running down your back tests, like back at university. {shudder}.

I lean towards “some just aren’t good at taking tests so let’s not” and he leans to “we must test to prove competence”.

Below is a long-ish article he’s written and sent to me in an email with the subject “Controversial post to consider”. I don’t know if it’s really that controversial, but it does put a question to us. Should we be testing for competence, not just assuming it.

Preamble:

After Ted presented to the APGA community in June 2022 on “Failure is Normal: A Tale of Two Bridges” (the Quebec Bridge and the Westgate Bridge) (link to APGA webinar here), he went back and reviewed the list of similarities between the two bridge failures, and he realised that this similarity in particular represented an important message for engineers:

The designer / consulting engineer reputations went unchallenged.

The engineering firms engaged in both cases were assumed to be competent by reputation alone, but that “competency’ was not proven before selection in either case.

Below are not my words, they are Ted’s. Let’s discuss.

===========================

Questioning competency assessment

By Ted Metcalfe, Independent Consultant

In the Quebec Bridge (1907 and 1916) and West Gate Bridge (1970) failure events, more than 100 construction workers were killed partly as a result of failings by design engineers, yet the competency of those engineers was neither questioned nor proven before they undertook their design work.

Would Registration of engineers have prevented the West Gate bridge failure?

In hindsight, no one knows for sure, but I doubt it. Here’s why.

In 2018, the Victorian government introduced legislation to register professional engineers. This was actually a direct consequence of failed government regulation of the occupied building industry over previous decades; however rather than admit failure, governments prefer to identify scapegoats.

Starting about three decades ago, building regulations were progressively modified to “encourage economic activity and create jobs”; and bureaucrats created “deemed to comply” interpretations of the Building Code; all of which collectively allowed non-compliant materials and dodgy practices by developers to proliferate.

I strongly suspect that even if the Victorian Professional Engineers Registration Act had already been in place when the West Gate bridge project was undertaken, the reputation of the design engineer was such that the government of the day would simply have declared that the firm was “deemed to comply” with the competency requirements of the legislation, and the tragedy would still have happened.

In Australia today we have a serious threat to public safety. Thousands of homeowners are stuck living in fire traps because of the flammable cladding debacle, and even more are struggling to get serious building defects rectified. 

The government’s answer? …………. Make it look like building engineers are to blame!

Who says whether or not (Pipeline) Engineers are competent?

For many years our industry-specific pipeline systems standard (AS 2885) has required that it only be used by competent persons. Looking back now, I realise that engineering competency demonstration and assessment has been an issue in our industry for a long time, and it is still not resolved to my satisfaction.

About twenty years ago, some industry experts conducted a number of seminars to emphasise both competency and compliance with the AS 2885 suite of Standards. 

They included a mock “court in session” role play to put individual engineers “on trial” for failure to comply with the requirements of AS2885; or for not being “a competent person” as required by AS2885.

In one session they picked me as the suspect, and of course I was judged by His Honour Haddow in the black robes and curly wig to be “guilty” of some crime we’ve both since forgotten.

However, I clearly recall responding to his sentence by demanding to know how the average pipeline engineer was supposed to determine or demonstrate competency as required by the Standard. 

I don’t recall getting anything like a clear and concise answer back then.  

How is engineering competency assessed now?

Competency based on the APGA PECS is currently assessed by a panel of engineers who review evidence submitted and interview the applicant. “Competent” or otherwise is essentially a judgement call.

The PECS originated back in 2008, when myself and others prepared a Business Case with a focus on training for pipeline engineers.

At the time, as part of our research I contacted Phil Hopkins, the eminent educator of pipeline engineers worldwide, and from my notes of our conversation it was his opinion that….

“…. competence can only be assessed by formal examination under controlled conditions with a clearly defined level of correct answers.” 

It made sense to me back then (and still does) that in order to demonstrate that you have learned something you must be able to prove under test conditions that you have indeed retained the information.

Our recommendation for development and operation of industry-sanctioned training courses with examinations was not accepted by the Association Board at the time. Instead, a project was undertaken to develop the competency documentation (resulting in 240 competencies, the Pipeline Engineering Competencies System), for commercial entities to use as they see fit.

So now we have defined competencies for pipeline engineers which is certainly a good outcome, but the PECS approach alone is neither complete nor effective in creating change.

Why is competency demonstration not mandatory?

In my APGA Convention paper last year, “Pipelines and Public Safety: How Reliable Are We?”, I pointedly observed that even with the PECS in place, competency demonstration is still not mandatory for engineers in our own industry……. Why not?!     

How should competency be assessed?

In my opinion, competency assessment should include an examination with a minimum pass mark.

I agree that competency assessment should also include a review of written submissions and an interview with a panel, but I strongly believe we should add a written examination to the process to ensure that the assessment includes a measurable component which is not simply a judgement.

Think about it………in order to obtain a driver’s license in Australia you must pass a written examination; but to practice engineering, as long as you have degree qualifications and some experience, neither written test nor license are required.

In his book “The Making of an Expert Engineer”, James Trevelyan makes some good points about competency assessment, and from the notes I made while reading his book:

A formal examination is a necessary part of training and motivates learning.                         

No exam means there is no motivation to really make an effort to learn.

The Canadian requirements for engineering competency assessment were prompted in part by the Quebec bridge failures over 100 years ago, and those still include a written multiple-choice Professional Practice Examination. 

Visitors wishing to access the workplaces of most major operating companies in our own industry are required to undergo a formal induction process comprised of an on-line review of workplace safety and other corporate policy information, followed by a multiple-choice quiz to identify whether or not they have satisfactorily understood the important relevant information. 

They must pass the quiz to be allowed on site, and usually must sign something to acknowledge that they have understood the information.

Can we implement exams for (Pipeline) Engineers?

We can. 

With our PECS already defined, much of the hard work has already been done; it’s just our competency assessment process that needs changing.

The pipeline industry must voluntarily and formally recognise pipeline engineering competency at least as highly as visitor safety in the workplace. 

If we do not, in the event of a serious accident our regulators may well impose upon us a legal “duty of care” to do so. 

As I said in my recent webinar, and as demonstrated above for the occupied building industry, engineers are an easy target for blame.

What do you think?

Surely someone else out there is willing to express an opinion on a matter this critical for our industry.

Please consider this carefully, and you are encouraged to leave a reply either supporting or objecting to my positions. 

If you are opposed to adding a written competency examination to the assessment; or if you do not agree that we should make competency assessment mandatory for pipeline engineers, please reply to join the conversation and explain your positions.

Even if you don’t have an opinion yourself, just forward this to someone else you know who should.

I always want to encourage debate about standards, ethics, and our industry. Every opinion matters.

Ted Metcalfe, Independent Consultant