Virtual Festival of Evidence | Prof. Mike Carter


Major challenges in hospital modelling, and tips for meeting them


Prof. Mike Carter
University of Toronto


Mike Carter: I’m doing work with the ministries of health in five or six different provinces in Canada, and I’ve worked with most of the hospitals in the Toronto area, as well as hospitals in Montreal, in Newfoundland, in Alberta. I think one of the big issues is forecasting health human resources – so, for example, I build models to figure out how many cardiac surgeons Canada needs for the next 20 years. I’m trying to do the same for cancer. The way people think right now in healthcare is to ask ‘how many do we need next year?’. But it takes 10 years to train people, so more work needs to be done on future forecasting. Getting people to accept the idea that we can do longer range planning than six months to a year is difficult.

Interviewer: How do you convince people that modelling is the solution to their crisis?

Mike Carter: It depends on the problem. Anything that I do I do with a champion; I’m working with somebody who already believes that I can solve their problem. I search those people out, and then we work together to try to convince others by emphasising ‘what is this model going to do for you as an individual doctor, nurse, administrator? How is this going to make your life better?’.

Interviewer: So you give them ownership of it?

Mike Carter: Yes. Ownership and a belief that I’m not doing something ‘ivory tower’ – I’m actually doing something that really is going to help them.

Interviewer: Is there an ‘aha’ moment when suddenly the penny drops and they can see what this model does and what it will achieve for their hospital?

Mike Carter: Yes, absolutely. I think that initially they don’t trust or appreciate the model, and the results don’t agree with what they’re doing. Then suddenly there’s a moment when they’ll say ‘I see, it is different. I understand what you’re doing’. Then from then on there’s an element of trust.



Okay, thank you very much. I was waiting for that to get set up before I can get myself set up. I’ve been asked to do this sitting down. I’m usually running around the room when I do things like this, but we’ll give this a try. Okay so I asked Terry what he’d like me to talk about and he sent me this title, Top Five Challenges in Hospital [0:03:00.0] Modelling, Tips for Meeting Them. I’ve been doing modelling in…my background’s operations research. But I’ve been doing modelling in healthcare for about 25 years. I’ve had over 100 projects. I’ve had some successes and some failures et cetera.

So I was going to try to give you…as I try to pull this thing together and give you a sense of my experience with what things are happening. So I’ve tried to change healthcare. Calvin and Hobbes, I thrive on change. You, you threw a fit this morning because your mum put less jelly on your toast than yesterday. I thrive on making other people change. Okay, so that’s mean. So as I heard around the table here you people are all familiar with the idea of models. One of the problems I face is that when I go and talk about models people have different concepts. Everyone thinks they know [0:04:00.0] what it means.

But they don’t necessarily mean oh our models. Of course obviously I’m focusing on things like simulation patient flow, forecasting, queuing, stock and flow modelling et cetera. I should add too that by the way this talk I’ve been doing this in the middle of the night and working on this for about three weeks. Going, what am I going to say? I’ve never done a talk. All of these issues I’ve thought about for years. But I’ve never actually tried to put it together into one talk. So this is a bit of a…if anybody’s got any comments, suggestions about how to fix this up. This talk is going to be perfectly ready in about two weeks, so I’ll come back.

Also as Mark alluded to too, I think we classify problems as being operational time frame, as being immediate three months, tactical or strategic. On those three levels [0:05:00.0] I think we actually have had a fair bit of success. This applies to healthcare and manufacturing private sector, private sector even more so. There’s been a lot of success in operations research at the operational level, less success at the tactical level and not much at all at strategic. So one of the things that I’m going to focus on is we have had people doing things like nurse scheduling, who works today? Advanced access with GP patients’ appointment scheduling, home care, Lean, yes right [unintelligible 0:05:33.5], Mark you’re absolutely right.

At the moment in Toronto it’s either a really good thing or really bad thing, about half and half between the hospitals and ED simulation, things like that. One of the things that I decided…like I said I’ve been doing this for 25 years and I think for the first 10 I was focusing on operational level problems. People would come and ask me, we’ve got a problem scheduling our docs or our nurses [0:06:00.0]. I was working on one problem, we spent about a year looking at transfer time between surgeries in an operating room…what can we do about reducing transfer time? So we spent a year, I had a couple of students working on it.

At the end of the year, they managed to save 10 minutes a day in transfer time in just how you schedule the operating rooms. I went wait a minute, I’m doing nickel and dime stuff in a billion dollar industry. I went, what am I doing? So I decided that I would like to work on strategic, on policy level problems, as opposed to the operational. Or I would like to work on operational problems, but not just one off. I’d like to do things that are generic. Of course it’s one thing for me to decide I want to do that. It’s quite another thing to actually pull it off. Because I can’t work on any problem unless somebody comes and says, Mike, we’ve got a problem, can you come and help us?

[0:07:00.0] So we’ll see what we can do here. So Terry asked me for five challenges. Well it’s easier to think of 20 or 30. Narrowing that down to five top ones is tricky. I pulled out a paper from Sally and Paul was on the list too and they used the technology assessment model from [unintelligible 0:07:23.1] and [unintelligible 0:07:22.0]. They were looking at all the different, what are the challenges for modelling in healthcare? So I went through that list and I pulled out some things. I said there are some issues that are serious and important. But there are things that the modellers can deal with. These are things that we have to do.

So usability, ease of use, security, demand analysis, deployment, pricing. Those are all things within our control. We don’t have to worry about the hospital. So I’m not going to talk about that because that’s not a challenge per se. There’s also external issues. So there’s regulation, [0:08:00.0] competitive pressure [unintelligible 0:08:04.2] effects and things. Those are also, those are things that are outside of the control of either the modeller or the hospital. We just deal with it. Then in that list from the paper there’s a whole bunch of challenges that are really focussing on challenges with the hospital, perceived usefulness, perceived fit.

The structure of the organisation, internal issues, personal characteristics et cetera. Again, I say there’s a long list and I think people could easily add to it, experience with innovation et cetera. So I try to do…I try to use the experience of things that I’ve worked on over the years and what were the challenges I hit? Then try to narrow this down to Terry said five. I think the title’s different now. But I like the idea of getting it down to five all encompassing. Please let me know if you think I’ve missed or you can think of other ways [0:09:00.0], other things to put in. This is a work in progress. So just some examples of some of the things that I’ve done with SIMUL8, with Claire’s gang, I developed a generic simulation model for OR planning.

So we developed a model that looked at how OR operating rooms…basically it’s a tactical level problem about which doc gets which room for the day or the half day. Then the impact on beds, on recovery rooms, on ICUs et cetera. So what’s the downstream impact, plus the upstream impact on things like waiting lists et cetera? We developed a generic model. I decided you should be able to create one model that any hospital and go in and say, in my hospital I’ve got 10 ORs and here’s the docs and here’s the types of patients et cetera. As I usually do, I thought that’s a great idea, I turned to a PhD student and said, why don’t you go do that?

Five years later she did. We’ve now [0:10:00.0] implemented this thing in 10 hospitals. For the first three or four we kept finding things we hadn’t done. By the time we got to the five or six, everything that we needed to do in our model…so we’ve done large teaching hospitals, small community hospitals, rural hospitals et cetera. It was a real challenge to get…generic is not as easy as it sounds. I also…I just presented this in Wales and in Southampton last week. Another PhD student…I was asked by the Ministry of Health…they wanted a model of the healthcare system. After I stopped laughing, I turned to my PhD student and said, why don’t you do that right? They hate me, but whatever.

They love me, they hate me. He did, he developed a model of the healthcare system. Anybody who is interested, I’ll show you the picture. It’s really astounding. I’ve done things for health human resource forecasting. Like how many cardiac surgeons does Canada need for the next 20 years? Locating thoracic centres around the [0:11:00.0] province of Ontario, which has about 13 and a half million people. Surgical wait list modelling. How many surgeries do you have to do to get down to targets? Ambulance policy analysis, a number of ambulance policies, capacity planning. So I’ve hit the whole gamut of types of problems and as you can see I’ve got things that are really tactical, strategic, policy models, going five, 10, 15 years into the future.

Whether they’re right or not is another story. But that’s neither here nor there. So here’s my five, my five things. The first thing is in order to be successful…now I initially said you needed to have burning platform. Talking to Sally, I said a burning platform and they looked at me and said, what’s that? So I said you need a crisis. Something that…you need a problem and I like to think what is keeping the CEO up at night [0:12:00.0]. Now what are they worried about? As Sally and the gang said in her paper people are always concerned they just don’t have the dollars, they don’t have the time, they don’t have the [unintelligible 0:12:10.2].

Yeah, yeah we can do that Mike. That’s an interesting little problem you’re working on. I need them to say we really need this and your model is the solution to my crisis. If that’s not there then we’re going to be in big trouble in terms of actually trying to get people and trying to get time and trying to get collaboration. We also need them to believe that I can solve their problem, that I can actually address the crisis that they’re after. So I put that as my number one. We need a crisis, my health system, I was asked to build this model of the health system because it would really be nice if we had a big investment to be able to use this health system model to decide what the impacts were.

I’m still [0:13:00.0] waiting for them to come up with what is the question to which this model is the answer? The health system gave me $150,000 to build this model, I built it and then went, gee, that’s nice. But nobody’s excited about it. They all look and it and say, yeah that’s really cute Mike and well done. Nobody’s using it. So it’s really frustrating. On the other side in terms of a crisis I had with my OR, the SIMUL8 project with the OR planning model, I was asked to look at a hospital in northern [unintelligible 0:13:36.1], Victoria Hospital and they had a critical issue. They were told that they had to get the wait times for orthopaedic surgery, their wait times, the ninetieth percentile was a year and a half.

They had to get it down to three months within three years. The hospital said, no problem, we can do that. We just need $2 million, a new orthopaedic surgeon and a new operating room [0:14:00.0]. The Ministry looked at that and said, we don’t think you’re running very efficiently. So the Ministry called me to take my model and go to Victoria hospital. You can imagine how welcoming they were. Actually they were pretty good. They were really pretty good about it. But it took a while to convince them that I had the solution to their problem and that I wasn’t just there from the Ministry to beat them up.

We got in and it turned out (a) they were very inefficient. They had to change what they were doing. (b) They really did need the extra resources and (c) turned out that there were some other things that they were doing that they actually had to change. One of the things that we discovered was that there was a surgical ward that had 32 beds and they told us regularly 22 medical patients were in those beds. They actually only had 10 to play with. In our model we were able to figure out that if you ring fence 17 beds in that surgical ward, give medicine the other 15. But you needed 17 or you weren’t going to get your numbers, right.

So we were able to use the model [0:15:00.0] to back that up. So in the end of the day…I mean it took for months. But in the end of the day the hospital was happy, the Ministry was happy and they got their $2 million. Second thing I put on my list is you need a strong champion. You really need somebody on the inside and I’m jealous of the NHS, I’m jealous of Australia. A lot of the things you guys have, that I don’t have. US, depends which system. There’s…really jealous of Kaiser and Intermountain and [unintelligible 0:15:30.9] and up and down [unintelligible 0:15:35.1]. But you really need a strong internal champion.

I don’t have any projects that I do without a clinical partner, without somebody in hospital, somebody beside me. I need a strong champion and I cheat. I said this is one thing. Well actually it’s a whole bunch, a strong champion who is actually also support from senior management. Plus somebody often in management [0:16:00.0] or clinical [unintelligible 0:16:01.2]. You need a strong internal team of managers, clinicians, analysts, who do the hands on modelling. I can’t just bring in a bunch of students who don’t know their way around the hospital or their way around the data and expect anything to happen.

Of course there’s always a risk of relying on a single champion for a long term engagement. For short term that’s fine. But there’s just a lot of mobility in healthcare. One of my favourite examples, we had a project with…we were going to study 10 emergency departments across Ontario and we built a simulation model for 10 emergency departments. We put in a major grant. We got $200,000. We got 10 hospitals to agree to have us come in with our students and do the modelling. After they agreed we went and wrote the grant. Actually I wrote part of the grant from [unintelligible 0:16:52.9]. We wrote the grant, we submitted it and six months later we got okay [0:17:00.0].

A year later the money started and we had everybody set up. We went back to the 10 hospitals. Five of the hospitals had no idea who we were because the manager who had agreed to this was gone. 50 percent turnaround in 12 months for ED management and it was just absolutely astounding. So that was the worst case. In terms of support from senior management, back in 2004, this woman Teresa Smith, who was the director of quality for Hamilton Health Sciences. So Hamilton Health Sciences, Hamilton is down the road from Toronto. It’s a city and Hamilton Health Sciences encompasses five hospitals.

A children’s hospital, a cancer hospital, a couple of general hospitals et cetera. Teresa saw me give a talk on simulation modelling and she said, Mike, this is really cool. We need to get some. We want some. So she invited me up to the hospital to give a presentation to, wait for it, the CRUM committee, C-R-U-M, which is the Committee on Resource Utilisation [0:18:00.0] Management. I think it’s a poor choice of acronym but never mind. So Teresa invited me up and then she really wanted to do some…she had no simulation background. She really wanted to do this stuff. So she actually had a competition within the hospital.

She had six different departments who were coming and doing a presentation to me on what they could…if they could be the ones who would be the pilot for simulation modelling right. So then I picked the one that I thought was the best fit. One of the projects was they had three cardiac ORs and they were building a fourth at one of the hospitals. They had to redo the OR schedule completely. They were trying to figure out how they were going to do that. But they weren’t building any new beds, they were just adding extra surgical capacity. So what’s this going to do downstream to the hospital?

So we took that and that was our model. Since then Teresa is now the CEO of the largest hospital within Hamilton [0:19:00.0]. She’s created this quality team. They have hired five of my students over the years, they’ve created a very strong little model. Unfortunately there are now a [unintelligible 0:19:07.7]. I still go there regularly. They’re really good. Another champion I got was a woman by the name of Janice Saner who is the wait time’s lead for the province of Newfoundland, which is also a little way from Toronto. So Janice has been…we’ve had many projects with her. With endoscopy clinics, colonoscopy wait processes, children’s hospital et cetera. So a whole variety of different projects.

Janice is just really keen on it because she’s keen, she brings me in and I just do what she tells me to and bring in students. Some of the challenges we’ve had. Yeah, so for example, clinical champion. So I have one of my PhD students on this one rarely. Most of my projects are funded by the hospital [0:20:00.0] or somebody. Rarely I actually get research funding to fund somebody. I decided in the province of Ontario, family practice, locations and specialties are just random. They throw money at docs, they’re private. So the docs get money for setting things up. They put up a [unintelligible 0:20:17.7]. No one’s really looking at the demand capacity, allocation, location of family practices.

So I started looking at this. I got this PhD student and I went and talked to somebody at the Ministry and said…somebody who was in that area and I said, would you like this stuff? She said, yeah, that’d be great Mike, let’s go. So we got it set up and Anna started it up and the next time I called her she’d been moved to another department and didn’t care about me anymore. I was so sad. Anyway, so it’s…I have still to this day, I can talk to people. They say yeah, it’s interesting, but I’m too busy. It’s not I can’t get a champion. I cannot get a…and it is a crisis. It really is. Family practice, GPs [0:21:00.0] is a crisis.

Nobody has been able to recognise that I might actually be able to make a contribution to the crisis, so they don’t get excited about it. So the third one I put on my list was good quality data. That’s ridiculous and hell, I’ve never had that. I’ve never actually encountered good quality data. If you’ve ever seen some, let me know. I’d love to hear about it. So with good quality data and I have, I get…when I go into hospitals and I walk in one of the first things that happens is they tell me…I say I need data and they say I’ve got tons of data. We’ve got data everywhere right. Have you got this? Pause. No, we don’t collect that. Oh shoot.

Then I realised what’s happening here, what’s going on is that the hospitals collect data because the clinicians need data information on patients. So they’re pretty good about that. They collect data because the bean counters need it [0:22:00.0]. They collect data because the government asked for it. They collect data blah, blah, blah right. So there’s a whole bunch of people who requested data, so they do it. I realised that nobody’s collecting data on patient flow, on continuity of care. We’re all talking about it now. But you guys in the NHS are actually a lot better than we are. We’re really very siloed in terms of data.

We just don’t have the link data, we don’t have the overall flow et cetera. I realised that it’s my problem that I keep blaming people because you don’t have the data I need. I realise it’s really me right, it’s us. As modellers we have to make it perfectly clear to people what data you should be collecting. Everybody’s interested in patient flow but they don’t know what to collect. We have to tell them. So we have to do things like minimum data set, information for patient flow, for understanding continuity care. What do you really need and what has to be connected, right. So those are the kinds of things and [0:23:00.0] that responsibility is really back on us. Garbage in, garbage out.

Yeah, people look at my models and say, well that’s the wrong answer and say, well yeah, but look what you gave me. That’s happened in many cases. Many hospitals that I deal with, they have high level summary data, data warehouse. So they can go in and find out [unintelligible 0:23:24.1] patients who’ve been here what they’ve done et cetera, episodic level. But in fact when I go in, when I’m doing my operating rooms planning for example, I want to know very detailed about patient in the room, patient out of the room, scheduled time, actual time et cetera. That stuff is in the OR system, but not in the hospital system.

Then there’s the nephrology has their system and cardiology has their system and ED has their own information system. They’re all sort of connected. If I want to pull out data even within just in the hospital I have to do the data request to link all these things together [0:24:00.0]. Pull out a picture of what happened to the patient in the hospital really at a detailed level. Summary I can get pretty easily, but that doesn’t normally help me with my models. So there’s a lack of link data, a hospital wide data warehouse lack of detail. Another one I had too is I had a major project on looking at ambulance offload and ambulances picking up people all over the city of Toronto.

So we were looking at, can we take instead of…I mean in the NHS you guys are doing some clever stuff that I’m trying to get them to imitate now for having…sending out somebody to treat somebody on the scene instead of taking them to the hospital. What we were looking at is we were looking at taking somebody to an urgent care centre instead of a hospital emergency department. Who could you do? We were trying to model who could you pick? What rules could you use and did you go to the right place? If you went to the wrong place, you’re in trouble. One of the things we discovered is the [unintelligible 0:24:55.5] data doesn’t talk to the hospital data. So we ran through [0:25:00.0].

I got de-identified data, which didn’t help. So I had to do probabilistic match. I got data for five years’ worth of every ambulance pick up and delivery in Toronto. I got five years worth of every emergency department, person who showed up in an emergency department. Then I did well if this ambulance dropped off this person around three o’clock at this ED and the ED said they picked up somebody by ambulance at this time, they’re probably the same person, good. We got about 95 percent match. But that’s the kind of crap I have to go through to try to…and I also need internal technical people who really understand the data.

I find this a lot in healthcare that the data people will sit there and they’re computer people, they’re IT people. Tell me what report you want, what field you want and they’ll dump it out. But they really don’t understand what they’re dealing with. They don’t understand the complexity of the data. They don’t have information, they have data bytes. If you want some data bytes I’ll give it to you [0:26:00.0]. Wait a minute this doesn’t really make sense, never mind. So there’s a lot of issues around data. Then one of the examples with my OR simulation model, I go in with my OR simulation model, they give me a pile of data, they say this is what we do.

So then we run the model and when we run the model, we discover that in our model…well I have one in particular hospital for sick children, it wasn’t this model, but it was 20 years ago. We went in and we did our OR model. They said here’s the time we’re doing for ENT, for ear, nose and throat and at that time 20 years ago the hospital for sick children was doing a lot of tonsils and [unintelligible 0:26:39.0], okay, 12 minute surgery. So our model went through and we said you’re doing 8000 a year. He said we’re only doing 5000, there’s something wrong with your model. The guy who is doing it went back to his office and he went through the code and he was going through and he went wait a minute, it’s 12 minutes, here’s the surgery time and that adds up to 8000 [0:27:00.0].

They said no, you don’t understand. You see some are longer than others. You don’t understand averages. I got a degree in averages okay. It turned out what was happening was the ENT docs were not using anywhere near all their time. They had all this time booked up. There was general surgeons who were starving. So as soon as the ENT doc didn’t need time, the general surgeon popped in. The OR manager figured the place was being used all the time. So based on just…you could have done this on the back on a napkin. I didn’t need a simulation model. The ENT doc, the ENT actually lost 40 percent of their time. It was that bad, it was that serious, that outrageous. So they give us this stuff and there’s something wrong with your model, no.

So in my simulation model now with the SIMUL8 stuff, we actually have a pre-processor. They give us a pile of data and say, well based on this data this is how many cases you’re doing and how many…this is how many OR hours you’ve got and what the average length of time is. Does that make sense? Before we bother running the simulation we do a lot of data [0:28:00.0] validation. We get outliers, the little old lady who was apparently there for five years as an impatient. We randomly select patients and recycle them in our models. So if we happen to get that little old lady, we’ve just screwed [unintelligible 0:28:13.7] for the time.

We typically use real patient data and the quality is not so great. The fourth one on my list for challenges for hospitals is healthcare culture. Healthcare culture, you guys know all this. But there’s a lack of a decision maker and I often facetiously say that if I went to the President of General Motors…I don’t want to pick on any particular company. But I went to a private sector company. I went to the CEO and said, I can save you a million dollars a day. If they believed me that I could do it, it would happen tomorrow. They would just…okay, we’re changing what we’re doing. This is what we’re doing. But if I did it to the CEO of a hospital, they would say we’ll have to [0:29:00.0] talk about it.

CEOs are not in charge. CEOs are typically facilitators. They’re trying to get everybody…yeah, this is a great idea, let’s talk about it. I used to think operations research in the private sector. Operations research, we are trying to optimise something. So minimise cost, maximise profit, maximise flow through et cetera. I realised in healthcare that’s wrong. What’s happening in healthcare, it will not happen, unless it’s actually better for the docs. It will not happen unless it’s better for the nurses, better for the bean counters, better for the administrators, better for the public opinion, media, government, decision support et cetera, perception.

It just won’t happen. I had one case where my system was going to make the nurses work an extra 10 minutes a day. The system didn’t work and nobody knows why. Okay…I think I…now the good news is the healthcare is so bad that I can always make it better for everybody [0:30:00.0]. Like let’s hire an extra nurse. So what? We’ll make it better for everybody. But I always have to look for that. So when I’m going through I say well here’s a great answer to my model. My model said this is what we should do, okay. So what’s this going to do to each one of the major stakeholders? You’ll notice I left off patients. I find it annoying.

Everybody talks about everything is patient focused et cetera. But really, deep down inside, not too worried about patients, anyway, but neither am I. But you can always go through it. I’ve got examples, like in my operating room scheduling, we’re trying to balance the demand for beds over the week. In order to balance demand for beds over the week, I’d like some docs to move around. Well docs don’t like moving around. So in my model I go through and say well if you move around, we can actually reduce cancellations. Well in Ontario, docs are fee for service. So if you get a cancelled surgery, you don’t get paid. So docs [0:31:00.0] really understand that.

Now unfortunately I want Doctor Jones to move so Doctor Smith doesn’t get cancelled surgeries, that’s another issue. They can fight about it. But you see you have to turn the dialogue into a thing where it is how is this better for you or is it worse for you and if it’s not what can we do about it right. So I think that’s important. Then an example, currently I’m doing some work in Winnipeg. So we came up with orthopaedic surgery, hip and knee surgery. One of the things that drives people crazy is that you go into the specialist and the specialist says, yeah, you need a hip replacement. We’ll call you. Then you sit at home for six, eight months waiting for them to call you.

Every now and then you call back and say what the heck’s going on right? It goes back and forth like that. So wouldn’t it be nice if you could say…I said the reason they’re doing that is because this is [unintelligible 0:31:55.9]. In the meantime there’s urgent emergent stuff coming in [0:32:00.0]. The urgent emergent stuff they need to save space for it, they don’t know how much to save. So they like to leave it until the last minute before they call in the electives. Well okay, I know what’s going to happen in the next few weeks, so I’ll call you up. Meanwhile the poor patient can’t do anything with their life. They can’t leave the country, they can’t leave the city.

If anybody told me I was going to have surgery in two weeks, my calendar’s a little full. So we’ve developed this Excel tool that’s based on a queuing model. It goes through and basically what we’re doing is we are estimating the probability of how many urgent slots should you save each week and et cetera. So we tell the patient we think it’s going to be six months. So right now okay it’s October. So your surgery is going to be in April, we’re not sure when. Then in January we’ll call you up, it’ll be the second two weeks of April. Then getting closer, it’s going to be this week et cetera. So we’ve got a tool that goes through and says, we’re 95 percent sure you’re going to be in this range [0:33:00.0].

I mean we think this is terrific. So the docs loved it, the nurses loved it. Everybody loved it. Okay, there are scheduling assistants who make the appointments, they would actually have to enter some extra data. They are bottlenecked, we can’t get past it. So they’re going we have to try to convince them that if they do this extra data then it’s going to reduce the number of phone calls coming back in. I haven’t heard from you, what’s going on right. So it’ll actually reduce their workload. But they haven’t seen it yet, so it’s really hard to…anyway, life goes on. Of course there’s the silo mentality in healthcare and the culture that…and I understand that, like the system is based, financially it’s siloed, organisationally it’s siloed.

Even within a hospital. People in a ED, healthcare is complex. So the people in the ED don’t actually have a clue what’s happening in the wards. The people in the wards don’t have a clue what’s happening in home care, long term care. Other than the fact I send it out there and those [0:34:00.0] people are not cooperating. I had a student several years ago and she was at one of the big cancer hospital in Toronto. The VP asked us to go look at this one department that everybody knew was dysfunctional. Everybody in the department knew it was dysfunctional. We went to the department and the department was thrilled that an industrial engineer was going to come in and help them fix it, help them make things better.

She got in and they were all really excited. She came to me about two weeks later in tears. She said, everybody wants her to make everybody else smarten up. They were nurses, they weren’t docs. I realised that that’s not that department, that is healthcare. Everybody is working 110 percent. I’m working my tail off here. Why isn’t everybody else cooperating? I haven’t got time to worry about you, but you’re not helping me. So a lot of that 110 percent is actually wasted effort and wasted time because we don’t have a flow, we don’t have a [0:35:00.0] balance and et cetera. That’s just within one department.

When you look at the whole healthcare system, it just gets worse and worse. The other thing I discovered too, several years ago I had one of my masters students was actually a person who had been a GP for…still is…a GP for 15 years. Just chatting with her back and forth and I discovered one of the things that happened is…I always look at the system. I always look at what are the performance metrics for the system and flow metrics and things like that. GPs are trained to think of individual patients. That’s drilled into them. Except for the public health people, the GPs, the docs are typically trained. The person in front of you is everything right and for 10 minutes and I have to come up with a solution in 10 minutes.

I have to do it in my head. I can’t go off and do some work on it right. So it’s really, they’re not really concerned with what happens outside of that because I’m only here for 10 minutes, you’re out of the door and [0:36:00.0] the next person’s here. I’m not really worried. I can’t really spend a lot of time worrying about what else is going to happen when you leave, who else you’re going to see or what else is going to happen. I understand that. It’s a huge challenge for us in modelling. The other thing is too that clinicians are anchored to randomise clinical trials. What I mean by that is that I walk in and they say, let’s do your simulation model at one hospital and not at another and compare…wait a minute.

What I do is I do before and after. I say, well this is what you did last year when you put in my model. I did a pilot we did for five weeks, things got better. It should be good enough evidence. It’s not for many docs. They’re so anchored to RCTs that they don’t get model evidence. They don’t understand what we do. So I think getting people to move to other kinds of analysis and other kinds of ideas is really important. The other things about the culture is that…and Mark mentioned it too, is that modellers [0:37:00.0] often try to emulate what they did in manufacturing. They just try to say, they’ve had a job working on the assembly line and they come in.

The problems that healthcare has are identical to the problems in healthcare, the differences, the culture. People come in and they have no clue about the culture and they say, I don’t understand. Why don’t you make those docs do this? It’s a sensible thing to do, right. Good luck with that chuck. There’s also too, there’s a reluctance to share too. Within healthcare I get people…we did some work with Alberta and we were looking at locating GPs in Alberta and what the demand, trying to actually measure unmet demand and look at GPs et cetera. I wanted to publish it. They said, wait a minute, you can’t publish it, it’s ours.

Wait a minute, a lot of people would really like to know about this [0:38:00.0]. No, we’ll have to talk to the deputy minister, we’ll have to talk, blah, blah, blah. So it took two years to get approval to tell anybody about what we had done [unintelligible 0:38:06.4]. It wasn’t like anything was bad. It was just a principle of getting it through the legal beagles et cetera. They were all like…they were horrified that I would…they can go tell people. But they were horrified that I might. Of course if you’ve seen me now for the last hour you can understand why they might be a little worried, but never mind.

The fifth thing I put down was managing expectations. People generally don’t know, don’t understand what we mean when we talk about models. They all think they understand models. Yeah, one of my favourite ones is that when I do forecasting models and I’m telling you what’s going to happen in five years or 10 years, there’s a long list of assumptions behind that. If all those assumptions are true then [0:39:00.0] this is what’s going to happen. I’m not doing predictions. People come back and say, wait a minute, you said this was going to happen in five years, it didn’t happen. People don’t understand scenarios.

They don’t understand. They somehow think that forecasting is magic. You can see the future like the old Johnny [unintelligible 0:39:22.4], never mind. You can see, predict the future. But we’re not predictors. We’re looking at trends and we’re looking at policies and if this and this happens, then that’s what will go on. So people are really looking for predictions for the future. It’s just not going to happen, okay. So I spend a lot of time getting through and we’re looking at scenarios if this happens, this happens, these are the kinds of things that happen. Then I ask them to make their best guess about which assumptions they think are going to be reasonable. I also get some issues with…a lot of people get stuck on, [0:40:00.0] they understand linear regression.

So linear regression is a forecasting tool and they get stuck on it. I had one problem, one woman who was doing some work…to be honest, to be fair she didn’t have a lot of experience with modelling at all. She took a bunch of data for nurses in Ontario and she did a linear regression model and she predicted that within 25 years every person in the province of Ontario would be a nurse. Okay, that is when they called me to their credit right. What happened with that was in the 1990s we had this huge cut in the numbers of nurses. So it went way down in terms of nurses per patient.

In 2005, 2010, we’d been doing huge…trying to get back and trying to get back capacity back. Well if you do a model based on that, it’s policy based, not demand based right. A lot of people of course are doing utilisation based models as opposed to demand based models [0:41:00.0]. Utilisation based models, this is what we have the capacity for and doing predictions on that doesn’t make any sense at all. So managing expectations is hugely important and I think that people do very funny things with it. The other thing is there are a lot of bad models out there. When I go and I try to tell people, yeah I can…there’s one in Ontario in the early 1990s where somebody built a model for how many physicians Ontario needed for the next 20 years.

It had a pile of assumptions and some of them were wildly wrong. It was a linear regression type model. But independent of whether the idea was any good or not. I’ve had many people to this day who come up and point to me, oh models are stupid. Look at that one. So they won’t talk. They completely dismiss all of us with a broad brush because they’ve seen a bad model and everybody’s seen a bad [0:42:00.0] model. They’re all out there. So people’s expectations along that line are [unintelligible 0:42:05.9]. I have a particular issue of course because a lot of my models are developed by students. Therefore they do not have the commercial grade user interface.

They’re sitting there, they’re clumsy and they’re clutzy. We’re trying to test out a theory et cetera. So they have expectations about oh well…they’re used to nice interfaces et cetera. So when I come along with my stuff they go oh, not so sure. One of the other things that they do is that people want simple solutions to complex problems. For example, my model of the healthcare system, I built this huge model of the healthcare system, a few of you have seen it. I think when they originally asked the question what they wanted was to be able to take the model of the healthcare system and put it on the desktop of all the policy analysts.

So they could just go and play with it [0:43:00.0]. Yeah, all right, it’s not going to happen. We really need…anyway. I think people have that expectation. I had a similar one with Alberta when we were looking at forecasting health human resources in the province of Alberta. What they wanted to do is they have a bunch of analysts working in health human resources in the province. They said, they wanted to have everybody’s desktop to have a little thing they could play with. So I think they think we really have to…what are you going to do with it? How are you going to use it? What’s it for? What’s the purpose? What’s the question we’re trying to answer?

A lot of people it’s just like a toy that they can play with to their heart’s content. They’re asking a lot. They’re asking a great deal. Modelling is an [unintelligible 0:43:46.0]. You’ve all seen these things. All models are wrong, but some are useful. It’s absolutely true. I drive this into people that we have to do….make your theory as simple [0:44:00.0] as possible but no simpler, I like. Then the model on the desktop here, Einstein said, for every complex question there is a simple and wrong solution. We see that a lot okay. Sorry Terry, I had a sixth one. The sixth one is credibility. I’ve really already talked about that. People have to believe that our models are actually going to solve their problem.

I think that there’s a lot of…first of all, nobody wants to be first. I want you to tell me that you’ve actually done it at 10 places before I’ll use it. So that’s a real problem and a real issue of the work with the hospital. The other is that you’ve done it at 10 places, but they were different. So your model probably won’t work at our place. Well, yes it will. So I feel like I tell people [0:45:00.0] I got a degree in Mathematics 35 years ago and 20 years ago I stopped doing mathematics. All I do now is marketing. I’m trying to convince people, please, please, please use our models. So thank you very much and I’m looking forward to comments, suggestions an hour later.

Male: Thank you very much indeed. That was terrific. In fact this morning could be a pivotal time. I know healthcare isn’t all about hospitals. But a lot of the focus is on hospitals and it’s easier to model hospitals than it is to model other bits of the [unintelligible 0:45:43.4]. So thanks very much. Later on we’re going to have a talk from Daryl at the Mayo, telling us a little bit about some of the operation challenges. The Empire Strikes Back and we’ll get some balance there. But I think it’s really important. One of the things I want everyone to be trying [0:46:00.0] to do is to think what are the key messages that are coming out of here? I think actually Sally you’ve got your five chapter headings for the report. That’s probably going to be really helpful.