« An Unexpected Benefit of Computer Games
Finding My Limit »

Fixing an Unintended Consequence

Today I implemented a fix that will hopefully avoid a repeat of an incident that happend last week. It is the sort of problem that I enjoy to work on. Why isn't this doing what we would expect? And more importantly how do I get it to do what I want it to?

We have a scheme on the feed system to a unit that is designed to ensure that the feed pumps maintain a minimum flow at all times. If a the pumps run with low flows, they can get damaged. To avoid this, when the unit is operating at low flows, a control valve opens and recycles extra liquid back to the feed drum.

To work out how much the valve needs to be open, the flow of liquid through each pump is measured. These three flows are then added together to get the total flow. Logic counts how many pumps are online and therefore what the minimum flow should be. A standard PID control loop then moves the control valve as required to keep the total flow at the required flow.

When a pump is offline, the flow from that pump can often read negative. To avoid running the online pumps below their minimum flow, negative values are filtered out with a function that picks the maximum of the measured flow and 0: MAX(0, flow). Unfortunately this is where the unexpected consequence comes from. When a reading from an instrument goes bad, a control loop will freeze the output and the valve will not move. The operator will get an alarm to tell them that the value is bad and the loop has frozen.

In this case, the value doesn't go bad. The filter instead uses the other value in the max function and assumes the flow from that pump is zero. Last week this happened and as a result the kickback valve opened in an attempt to increase the flow from the pump. This caused a minor upset to the unit unnecessary.

To avoid this in future, I have modified the logic such that if a flow from a running pump goes bad, the calculation is marked bad and the loop now freezes as expected. If the flow from an offline pump goes bad, the logic doesn't change anything, the system assumes that the flow from that pump is 0 and the flow control continues to operate as normal.

Go Top
« An Unexpected Benefit of Computer Games
Finding My Limit »


To comment on this article, send me an email

No comments yet.