The contract between the BBC and Siemens Business Services for the provision of technology services Review by the Compt rolle r and Audito r Gene ral p resented to the BBC Gove rno rs’ Audit Committee | 15 June 2006 contents summary 4 main REPO RT Background 9 Is the BBC well placed to meet its 10 service objectives? Is the BBC well placed to meet its 13 financial objectives? Are the BBC and Siemens taking the contract 16 forward through partnership working? This report has been prepared under Clause 10B of the amendment to the Agreement between the Secretary of State for Culture, Media and Sport and the BBC dated 4 December 2003. John Bourn Comptroller and Auditor General National Audit Office The National Audit Office study team consisted of: Stephen Callow, Richard Gauld, Vicky  Lewis, Ashley McDougall, Doug  Neal and Gareth Tuck under the direction of Keith Hawkswell. A link to this report can be found on the National Audit Office website at www.nao.org.uk For further information about the National Audit Office please contact: National Audit Office Press Office 157-197 Buckingham Palace Road Victoria London SW1W 9SP Tel: 020 7798 7400 Email: enquiries@nao.gsi.gov.uk © National Audit Office 2006 Apendic es 1 Methodology 19 2 Good practice references 20 3 Key service targets under the contract 21 Photographs courtesy of the BBC The cont ra ct between the BBC and Siemens Business Se rvi ces fo r the p rovision of te chnolo gy se rvi ces summary 1 In October 2004, following an open procurement competition, the BBC entered a 10-year framework contract with Siemens Business Services (Siemens) for the supply of technology services previously provided by the BBC’s wholly owned subsidiary BBC Technology Ltd. The contract was worth an estimated £1.5 billion over 10  years and the BBC expected it to deliver savings of £275 million over that period. As part of the deal, Siemens purchased BBC Technology for £150  million. There was a planned transition period from October  2004 to March 2005, with the contract formally starting on 1 April 2005. 2 This report examines whether the BBC is well placed to meet its strategic objectives for outsourcing technology services. While recognising that the contract is at an early stage, we assessed performance over the first year of the contract and considered whether the BBC is equipped to realise the anticipated benefits. We have considered three principal dimensions of the outsourcing arrangement: service delivery; financial benefits; and partnership working. Findings On service delivery 3 Our main findings are: n Siemens has delivered the level of service required by the BBC under the contract for its key service targets 95 per cent of the time. It has also met the other performance targets 90 per cent of the time. The performance measurement system was tested before each measure became live. Although the BBC looks at the data when it is submitted, to see if it appears broadly reasonable, and data accuracy is checked when a target is not met, there is no ongoing detailed validation by the BBC of the performance information supplied by Siemens. There are 376 performance targets in use and the BBC recognises that gathering and analysing the data to support performance information is costly. It is planning to review targets to check whether they remain relevant to its business needs [paragraphs 20, 21, 28 and 29]. In 2005-06, the BBC estimates it spent some £450  million1 to support a range of technology needs. The contract with Siemens accounts for some £190  million of that amount and broadly covers the services previously provided by BBC Technology, including: n provision and ongoing management of IT commodities, such as desktop computers and telephony services; n specialist technology projects; and n technological support for programme production and broadcast functions, for example to enable digital content development and editing, and uplinking programme content to broadcasting satellites. 1 The £450 million includes broadcast specific high value technology, such as studio cameras, and commodity items, such as video tapes, which the BBC traditionally buys directly from manufacturers. The cont ra ct between the BBC and Siemens Business Se rvi ces fo r the p rovision of te chnolo gy se rvi ces n The BBC is using the contract to engage in forward -looking technology transformation projects. As a framework, the contract provides the BBC with the flexibility to commission new work to reflect changing business requirements. There are also 41 projects identified during contract negotiations and specified in the contract that make up the Service and Technology Transformation programme. Five of these projects have since been cancelled by agreement [paragraph 24]. n As of May 2006, eight projects have been delivered, of which five were late and three were over budget. Of the 28 projects still under way, nine are delayed and 10 are over budget. Siemens bears the risk of cost increases on all the projects which have run over budget to date. However, delays in the completion of some projects mean that some of the intended benefits to the BBC of the Service and Technology Transformation programme will not be available until later than planned [paragraphs 24 and 25]. n Siemens also took over responsibility for delivering 68 ‘in-flight’ projects started by BBC Technology to timescales and budgets agreed between the BBC and BBC Technology. Fifty -four of the 68 projects have been delivered, or are on target to be delivered, according to time and budget and 14 have experienced delays or cost increases [paragraph 26]. n The contract contains two key service targets designed to measure BBC staff satisfaction with Siemens, both through monthly user surveys. One survey was first issued to 500 BBC staff in January 2006 but received only 24 responses. The other survey is still being tested. Although there are other means the BBC can use to build a partial picture of user satisfaction, after the first year it only has a limited end user perspective of the success or otherwise of the contract [paragraph 22]. 4 The contract incorporates a number of provisions designed to incentivise performance. These include: n a gradual increase in the level of performance required and a service credit regime which reduces the service charge paid by the BBC if performance falls below specified levels [paragraphs 30 and 32]; and n a performance benchmarking system (from year four onwards) for services which account for two -thirds of the estimated contract spend and for which the BBC and Siemens agree there are market comparators available. Under the contract, Siemens’s performance could be compared to the average of the top 50 per cent of comparable services. However, performance could be up to 7.5 per cent below this average before Siemens was required to improve [paragraph 31]. On financial benefits 5 Our main findings are: n In September 2004, when the Governors approved the contract, they were told it would deliver average guaranteed annual savings of £35.2  million. However, in practice the savings could not be guaranteed. Although the contract specified the unit prices and day rates to be charged for technology services, it did not provide a guarantee that a specific level of savings would be achieved. When the contract was signed in October 2004, the estimated savings were £27.5  million because the baseline spend (based on spend with BBC Technology) against which savings are measured was recalculated to remove £7.7  million of non-recurring expenditure.2 By April  2006 estimated savings for 2005-06 (the first year of the contract) were £21.8  million because spend under the contract was higher than forecast [paragraph 35]. n The BBC also expected to make separate annual savings of £3.9 million in 2005-06 by reducing demand for technology equipment through its redundancy programme. The redundancy programme was delayed while the BBC discussed the programme with its unions and the BBC now anticipates that the savings will be realised from 2006-07 onwards [paragraph 36]. 2 £35.2 million is calculated against a baseline cost of £190 million and £27.5 million against a baseline cost of £182 million. n The BBC is measuring savings by tracking cost reduction against baseline spend. However, measuring cost reduction alone may not give the BBC a complete picture of financial benefits over the life of the deal. For example, savings can also arise from purchasing a larger quantity, or improved quality of services under the new supplier for the same overall cost [paragraph 37]. n The BBC has taken steps to strengthen its financial management processes relating to the deal by obtaining more detailed billing information. It also plans to centralise control over project funding and employ technology service advisors to better manage BBC Divisions’ technology spend [paragraphs 40 to 43]. 6 The deal with Siemens does not include a provision to share profits if the rate of return exceeds a specified level. However, the contract does include the following provisions designed to keep prices competitive over the life of the deal [paragraph 44]: n Incentivised savings – Siemens is required to identify projects with potential savings of £81 million over the life of the contract. If proposals are unacceptable to the BBC, Siemens is not required to put forward alternatives [paragraph 45]. n A gain share mechanism – this stipulates that any net savings resulting from gain share projects are split equally between Siemens and the BBC [paragraph 46]. n Open book accounting – the BBC does not consider that there is a benefit to open book accounting in the first 18 months of the contract as the BBC and Siemens do not expect Siemens’s rate of return to be positive in this period. The BBC sees that there are benefits to open book accounting for new projects and has exercised its rights selectively in this respect. Otherwise the BBC has not used its open book access rights and does not know Siemens’s actual rate of return [paragraphs 47 and 48]. n Price benchmarking – as with performance benchmarking, this applies to two -thirds of estimated spend and prices could be up to 7.5 per cent above the average of those examined before Siemens was required to reduce them [paragraph 49]. On partnership working 7 Our main findings are: n The BBC and Siemens consider the partnering relationship to be constructive but the BBC recognises that greater engagement of Board-level executives is needed and has made plans to achieve this [paragraphs 51 and 54]. n There is a need to improve the level of understanding of Siemens’s role among BBC Divisions. Applying the contract to BBC Divisions which still provide some of their own technology services could extend the benefits of the contract [paragraphs 51, 56 and 57]. n The BBC intended to reward Siemens for high levels of partnership behaviour over and above routine service performance through a balanced scorecard. However, the two parties have been unable to establish an effective way of measuring high-level partnering performance [paragraph 53]. n For shared contract risks, a joint risk register is being produced, but the BBC has no separate register for the contract for those risks which rest with the BBC, although some risks are reported in its corporate risk register [paragraph 55]. n The contract contains provisions to support a re -competition when it expires, but the BBC does not currently have contingency plans in the event of early termination and recognises that these should have been in place from the outset [paragraph 60 and 62]. 8 It is clear that the deal is delivering benefits, however, it is too early to say whether the BBC will achieve the full value for money expected from the contract over its 10-year life. Savings are being achieved, though currently below the forecast level, and reported performance against key service targets is high, although some technology projects are currently not on track. The BBC could have made more use of the transition period to test and bed in the arrangements it needs to manage the contract effectively. Going forward there are measures in place which have the potential to drive value for money by promoting high service levels and reducing costs and the BBC has been working to strengthen these arrangements. Our recommendations are designed to help it in that process. 9 The BBC should: a Establish an up-to-date estimate of savings over the life of the contract, inform the Governors how this estimate has changed since the contract was approved and update the Governors if the estimate changes significantly in the future. b As well as measuring cost reduction, provide a more complete picture of the financial benefits of the Siemens deal by, for example: n estimating what the quantity of service provided by Siemens would have cost under BBC Technology while demonstrating that the service quality meets business needs; n estimating the impact of technology transformation projects. c Strengthen performance management by: n validating the accuracy of performance information provided by Siemens, on a sample basis to keep costs to a minimum; n drawing on FABRIC3 good practice when conducting its planned review of performance indicators, placing particular emphasis on the costs of maintaining, and the utility of measuring 376 performance indicators; and n emphasising to staff the importance of responding to user satisfaction surveys. d Improve the level of understanding of Siemens’s role among BBC Divisions and assess the costs and benefits of, and barriers to, increasing the take-up of the Siemens contract across all BBC Divisions. e Work more closely with Siemens to understand the reasons for project overruns and ensure effective action is taken to improve project delivery to time and budget. f Draw up a contingency plan to be used in the event of early contract termination. g Improve its risk management processes by finalising the joint risk register and producing its own separate risk register for the contract. Conclusi ons and r ec omm enda ti ons 3 Choosing the right FABRIC – a framework for performance information (HM Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics). Conclusi ons and r ec omm enda ti ons c on tinu ed Lessons for other outsourcing contracts Our review of the Siemens contract provides potential lessons for other outsourcing deals. The BBC should: 1 use precise language to express anticipated financial benefits and be clear about the probability that these benefits will be realised when asking for the Governors’ approval to outsource business functions; 2 update the Governors if the forecast of financial benefits is revised before the contract is signed; 3 where appropriate, engage senior management at Board level from the outset to underline the importance of the partnership and co-ordinate strategic priorities; 4 ensure from the start of the contract that information provided by the supplier is sufficiently detailed to allow the BBC to validate the bills; 5 introduce processes to validate the accuracy of performance information; 6 draw up contingency plans to be used in the event of early contract termination; 7 produce its own risk registers for contracts; 8 consider whether open book access provisions could be strengthened by securing access to income and expenditure forecasts as well as historical data; 9 implement open book accounting from the outset to protect its interests and embed transparency and accountability in working practices; 10 consider introducing a contractual provision to share profits if the rate of return exceeds a specified level, as one of a number of mechanisms to safeguard value for money over the life of a deal. Background 10 In October 2004, the BBC outsourced its technology function to Siemens Business Services (Siemens) in a deal which is the largest of the BBC’s outsourced contracts with an estimated cost of £1.5 billion over ten years.4 11 Prior to October 2004, the BBC’s technology services were either ‘self-provided’ or provided by BBC Technology Ltd, a wholly owned subsidiary of the BBC. In 2003, following a review of all the BBC’s subsidiaries and benchmarking of the prices charged by BBC Technology, the BBC concluded that selling BBC Technology and entering a 10-year outsourced contract would offer better value for money and better meet its future technology needs. 12 The BBC issued a notice in the Official Journal of the European Union in December 2003. The BBC received 31 applications and in February 2004 invited nine bidders to negotiate, of which three were shortlisted. Siemens won the competition because the BBC concluded that it offered a stronger technological solution, a better cultural fit with the BBC, a competitive price for the sale of BBC Technology, and savings from the start of the contract. Following approval from the BBC Governors, Siemens was announced as the preferred bidder in July 2004. Its final offer was to pay a cash sum of £150 million5 for the acquisition of BBC Technology, and deliver savings of £275 million, in cash terms, over the life of the contract. The contract was signed in October 2004. 13 The BBC Governors were satisfied that the competitive procurement process delivered value for money at the start of the contract, but asked the National Audit Office to examine whether the BBC is well placed to deliver the anticipated benefits over the contract’s 10-year life. We did this by assessing: n whether the BBC is equipped to realise the anticipated benefits of the contract in the future; and n early performance against the BBC’s objectives (while recognising that the contract is at an early stage). 14 Our approach is set out in Appendix 1 and was informed by good practice, set out in Appendix 2, drawn from reports on outsourcing by the Committee of Public Accounts and the National Audit Office, and guidance by the Office of Government Commerce. The Technology Framework Contract between the BBC and Siemens 15 The Technology Framework Contract is for the provision of technology services by Siemens to the BBC, the World Service and the BBC’s commercial subsidiaries. The contract with Siemens broadly covers the services previously provided by BBC Technology, and accounted for some £190 million of the estimated £450 million6 it spent on technology in 2005-06. The contract runs for 10 and a half years, from 1 October 2004 to 31 March 2015. MAIN REPO RT 4 The BBC’s other outsourced functions are property services, licence fee collection, transmission, playout (the stage between the production of a programme and its transmission which involves the packaging of programme content so that it can be broadcast to viewers), finance and call centres. 5 The net proceeds from the initial cash sum were £121 million following adjustments for inter company debt, tax, working capital, deal costs and separation costs. 6 The £450 million includes broadcast specific high-value technology, such as studio cameras, and commodity items, such as video tapes, which the BBC traditionally buys directly from manufacturers. There was a planned transition period from October 2004 to March 2005, designed to give Siemens and the BBC the opportunity to put in place the people, structures and processes needed to manage and run the contract. The contract, and the relationship with Siemens, is managed by the BBC’s Technology Group and formally started on 1  April 2005. 16 For some services under the contract, Siemens’s supplier status is exclusive, while for others it is preferred or non-exclusive. For a preferred service, the BBC must ask Siemens to bid for work before any third party. These arrangements are in place until 2010. For a non-exclusive service, the BBC may purchase from either Siemens or third parties. 17 The contract provides for a range of technology commodities, such as desktop computers and telecoms, and more specialist project work. Some commodity services have a fixed price while for others the unit price increases if volumes fall into a lower band and decreases if volumes rise into a higher band. Prices can be re-negotiated if volumes rise 40  per  cent above or fall 20  per  cent below contracted levels. For project work, the BBC either commissions work on a time and materials basis or agrees a fixed price with Siemens for completion of the project. The day rates for project work were arrived at through the competitive tendering process. 18 The BBC expects the contract with Siemens to provide a flexible and secure technology infrastructure and services to meet its business needs over the next 10 years. This vision is supported by a set of strategic objectives specified in the contract. These are to: n deliver emergent technologies and innovation; n maximise value for money and identify opportunities to reduce the service charges paid by the BBC; n accommodate changes to the BBC’s service requirements; n deliver excellent ‘customer service’; and n provide maintenance and support of a resilient technology infrastructure and continuity of services for a live broadcast environment. We have addressed these objectives under the headings of: whether the BBC is well placed to meet its service objectives; whether the BBC is well placed to meet its financial objectives; and whether the BBC and Siemens are taking the contract forward through partnership working. Is the BBC well placed to meet its service objectives? Early performance 19 When the BBC entered into the contract with Siemens, it took the opportunity to introduce new performance measures and change some of the existing measures to bring them into line with current business needs. Consequently, the level of service under Siemens cannot be compared with that under BBC Technology. However, the percentage of performance targets met has been high and BBC Management considers that overall service delivery has improved under Siemens. 20 Under the contract, there are 29 key service targets, 27 of which are currently in use (the remaining two relate to user satisfaction surveys which are covered in paragraph 22). The targets cover a range of requirements, such as helpdesk response times and internet availability. There is a full list at Appendix 3 and a summary of performance in Figure 1 below. Failure to meet these targets results in service credits which reduce the amount paid by the BBC to Siemens. 21 The contract also includes 446 other lower-level performance targets which cover different aspects of Siemens’s delivery, for example the speed of installation for desktop computers and fault resolution times. Of these, 349 are currently in use because some measures are not applicable from the start of the contract or are scheduled to be introduced in the third year. Failure to meet these 1 Summary of performance against key service targets in the first year n Performance against key service targets was measured 262 times. Siemens achieved targets 250 times (95 per cent). n Targets were not met 12 times, which resulted in the BBC receiving some £30,000 in service credits. n 100 per cent broadcast continuity for content platforms such as television and radio has been achieved, which the BBC sees as a key objective of the deal. Source: National Audit Office analysis of BBC data NOTE Not all measures were in use from the start of the contract, so the number of measures against which Siemens’s performance was measured each month has varied. lower-level targets does not result in service credits, although the contract allows the BBC to promote some of these to key service targets (to a maximum of 40). In accordance with the contract, the required performance levels were increased after the first six months. Since this increase in October 2005 the percentage of targets met by Siemens has fallen (Figure 2), but overall some 90  per  cent of performance targets were met in the first year. 22 One of the key objectives for the deal was to deliver excellent customer service to BBC staff. Under the contract, there are a range of lower-level measures relating to customer satisfaction, but in terms of key service levels, the means of measuring satisfaction is through two user surveys which should be distributed by Siemens at monthly intervals to BBC staff. One of the surveys covers satisfaction with the service helpdesk and was first issued to 500 BBC staff in January 2006 but received only 24 responses. The other survey covers call-off project support and is still being tested. Although the BBC can build up a partial picture of user satisfaction from other performance measures, the low survey response rate has made it difficult for the BBC to assess the overall level of user satisfaction with Siemens. The BBC and Siemens are now revising the survey approach. 23 The deal was intended to help the BBC meet its future technology requirements by giving it access to Siemens’s capacity to invest in new technology and innovate. Among the BBC staff we consulted, the consensus was that some innovation had been achieved in the first year but they wanted more going forward. Some of the investment in technology was written into the contract, but the framework provides the BBC with the flexibility to commission new work under call-off arrangements to reflect changing business requirements. 24 There are also 41 projects written into the contract which make up the deal’s Service and Technology Transformation programme, at an estimated annual cost of £4.5 million. The programme is intended to provide the BBC with technological infrastructure needed to support its business and the ability to move with changes in broadcast technology, for example multi-platform broadcasting. The programme is also intended to generate cost efficiencies and improve management information systems. Progress in delivering the Service and Technology Transformation programme in the first year is summarised in Figure 3 overleaf. Percentage of performance measures passed Month Source: National Audit Office analysis of BBC performance data 0 20 40 60 80 100 Apr Aug Sep Oct Nov Dec Jan Mar Feb Jul Jun May NOTE From October 2005 performance standards were increased. 95.2 93.6 95.3 94.8 95.9 94.4 87.7 88 88.2 88.4 86 87.2 Siemens’s performance against non-key service targets 2 25 Under the contract Siemens bears the risk of cost increases for 27 projects, Siemens and the BBC share the risk for three and the BBC has retained the risk for 12. As of May 2006 all ongoing projects for which the BBC holds the financial risk remain on budget. The delays in the completion of projects does mean, however, that some of the intended benefits to the BBC of the transformation programme will not be available until later than planned. 26 Siemens also took over responsibility for delivering 68 ‘in-flight’ projects started by BBC Technology to timescales and budgets agreed between the BBC and BBC Technology, subject to information provided by the BBC being accurate and complete. Fifty-four of the 68 projects have been delivered or are on target to be delivered according to time and budget and 14 have experienced delays or cost increases as shown in Figure 4. The capacity to deliver the BBC’s service objectives 27 Going forward, the BBC has a number of mechanisms to support the delivery of its service objectives. These include a new system to measure performance and a number of contractual provisions designed to incentivise service delivery, including a programme of continuous improvement, service credits and performance benchmarking. 28 The performance measurement system. One of the Service and Technology Transformation programme projects was to introduce a new performance measurement system to assess Siemens’s performance. With 376 ‘live’ performance targets there is a question of manageability and the BBC recognises that gathering and analysing the data to support performance information is costly. It is good practice to review regularly the costs and benefits of performance information systems7 and, as part of a planned annual review of the contract, the BBC has confirmed that it intends to review each target to check whether the target remains relevant to business needs. 29 The performance measurement system was tested before each measure became ‘live’. The BBC looks at performance data when it is submitted by Siemens to see if it appears broadly reasonable and, when a performance target is not met, the BBC and Siemens determine the causes of failure and check whether the data reported by the system is accurate. However, there is no ongoing detailed validation by the BBC of the performance data supplied by Siemens. 3 Progress in delivering Service and Technology Transformation projects Total To time Delayed To budget Over budget Both delayed and over budget Delivered 8 3 5 5 3 2 In progress 28 19 9 18 10 4 36 22 14 23 13 6 Cancelled by agreement 5 Total 41 Source: National Audit Office analysis of BBC data 4 Progress in delivering ‘in-flight’ projects Status of project Number of projects Delivered as at May 2006 33 According to time and budget 25 Either delayed or over budget 8 Projects under way as at May 2006 35 On target for delivery to time and budget 29 Either delayed or over budget 6 TOT AL 68 Source: National Audit Office analysis of BBC data 7 Choosing the right FABRIC – a framework for performance information (HM Treasury, Cabinet Office, National Audit Office, Audit Commission and Office for National Statistics). 30 Continuous improvement and increasing service targets. Under the contract, Siemens is required to prepare a programme of continuous improvements to services and to refresh technology. The BBC controls the rate at which technology is refreshed and has agreed that laptops will be refreshed every three years, desktops and printers every four years, and servers and local area networks every five years. In addition, performance levels for some targets increase from the third year of the contract (Figure 5). 31 Performance benchmarking. From the fourth year of the contract, the BBC can benchmark Siemens’s performance for services which represent just over two -thirds of the first year’s contract spend. The remaining one -third is not subject to benchmarking because Siemens and the BBC agreed that comparable benchmarks were not available. Under the contract, Siemens’s performance could be compared to the average of the top 50  per  cent of comparable services. Performance could be up to 7.5  per  cent below this average before Siemens was required to improve. 32 Service credits. When negotiating the deal, the BBC aimed to avoid a punitive service credit regime, recognising the need to balance a financial incentive for Siemens to meet targets, with a system that promotes partnership working. 33 Service credits are applicable to expenditure on commodities but not on projects. Each of the key service targets carries service credits and, in line with good practice, there is a limit on the maximum service credit available. The BBC and Siemens agreed the ceiling at 5  per cent of the monthly bill on commodities. Although the BBC recognises that the maximum available amount would not necessarily reflect the damage done to its business, it aimed for an amount that would be sufficient to significantly affect Siemens’s profit from the deal. To illustrate, the average monthly bill for commodities in year one of the contract was £11.68 million. The maximum amount of service credit the BBC could apply for this would be £0.58 million. Over the course of the year this would equate to £7.01 million out of a commodity spend of £140.19 million and a total spend of £190.32 million. Is the BBC well placed to meet its financial objectives? Early performance 34 The BBC expects the prices set out in the Technology Framework Contract to bring significant annual cost savings compared with baseline spend under BBC Technology.8 There are broadly two types of spend under the contract: spend on technology projects, and spend on commodities such as desktop computers and telecoms. When the BBC produced its estimate of cost savings, all of the savings related to commodity spend. 5 Some service levels increase over the course of the contract Key service level Year one target Year three target (per cent) (per cent) Number of projects delivered on schedule 90 95 Availability of bbc.co.uk 99.5 99.95 Availability of Radio 1, Radio 6, Radio 7, World Service 99 99.95 Percentage of broadcast critical incidents resolved 97 within 20 minutes 97 within 10 minutes Number of severe gaps in compliance with ISO1779 (security 2 0 standards) annually Source: BBC 8 When the deal was negotiated, Siemens and the BBC produced an estimate of future annual expenditure based on Siemens’s prices and existing levels of service. This estimate was set against comparable spend under BBC Technology (which became the baseline) and the difference between these figures became the estimate of future annual cost savings. 35 When the Governors approved the deal in September  2004, they were told that it would deliver guaranteed average annual cost savings of £35.2 million. However, in practice the savings could not be guaranteed. Although the contract specified the unit prices and day rates to be charged for technology services, it did not provide a guarantee that a specific level of savings would be achieved. When the contract was signed in October  2004, the estimated savings were £27.5 million because the baseline spend was recalculated to remove non-recurring expenditure. By April 2006 estimated savings for 2005-06 were £21.8  million in real terms because spend under the contract was higher than forecast (Figure 6). BBC Divisions’ technology commodity budgets have been reduced by £20 million to realise the benefit of the estimated savings. 36 In addition to providing more competitive prices, the framework contract allows the BBC the flexibility to reduce costs which were previously at fixed levels under the contract with BBC Technology (Figure 7). 37 The savings in paragraphs 35 and 36 relate to cost reduction which gives the BBC a basis for assessing by how much to reduce Divisional technology budgets year on year, and provides BBC Divisions with an incentive to drive down costs by controlling the volume of commodities purchased. However, measuring cost reduction alone may not give the BBC a complete picture of financial benefits over the life of the deal. For example, efficiency savings can arise from purchasing a larger quantity, or improved quality of services under the new supplier for the same overall cost. 38 As well as reducing technology costs, work commissioned under the Siemens contract is generating savings in other areas of the BBC. For example, the BBC expects that spend on travel and subsistence will reduce through the use of video-conferencing facilities currently being introduced by Siemens. 6 Reconciliation of the adjustments to estimated annual cost savings Source Approval papers to the Governors September 2004 Forecast savings when the contract was signed October 2004. Reductions in forecast savings since the contract was signed. Estimate of first year savings as at April 2006 Explanation Calculated assuming baseline expenditure of £190 million under the previous arrangement with BBC Technology. Although forecast spend under Siemens remained unchanged, estimated savings reduced because the baseline cost was adjusted from £190 million to £182 million to remove non-recurring costs. Calculated assuming baseline expenditure of £182 million under BBC Technology. Estimated savings reduced as spend under Siemens was higher than forecast. This was because expenditure which was sub -contracted under BBC Technology was not included in the original forecast spend. Estimated savings reduced further because the volume of commodities purchased under the framework contract was higher than anticipated. Average annual savings (£ million) 35.2 (7.7) 27.5 (4.5) (1.2) 21.8 Source: BBC Financial management 39 Since the contract was signed in October 2004 the BBC has been developing and improving its financial management processes relating to the Siemens deal to: n monitor the accuracy of Siemens billing; and n control the spend under the contract. 40 Monitoring Siemens’s billing. Both Siemens and the BBC recognised the need for improved billing when the first bill under the Technology Framework Contract was received in May 2005 and asked BBC Internal Audit to carry out a review. Internal Audit reported that the monthly bills from Siemens’ did not provide enough detail for the BBC to be able to check their accuracy and confirm that the anticipated cost savings were being delivered. We confirmed that, since October 2005, the BBC’s technology contract management team have been provided with more detailed billing information and that the BBC are checking the accuracy of Siemens’s early invoices. 41 Controlling spend. The BBC currently operates a decentralised budgeting system, which means that BBC Divisions are responsible for managing their own technology spend, some of which is outside the contract. The BBC does not know the exact value of projects commissioned by Divisions outside the contract but estimates that total technology spend was around £450  million in 2005-06 and £190 million of this was paid to Siemens. The BBC anticipates that savings could be made by procuring some of the products and projects purchased from other suppliers through Siemens. These savings could arise from more competitive prices under the contract or from lower administrative costs from not having to go through an additional procurement process. Expanding the coverage of the Siemens contract is also covered in paragraphs 56 to 57. 42 The amount spent in 2005-06 on new projects and on commodities under the framework contract with Siemens was £178.7 million,9 some 14 per cent more than the £156.7 million forecast. The BBC has explained that this was principally due to a greater number of projects being required as its technology requirements crystallised and new technologies (such as high definition television) becoming available. 43 The BBC estimates that additional savings could be made through more rigorous volume control and is appointing a team of Technology Service Advisors to help BBC Divisions better manage commodity spend.10 It is also planning to transfer control over project spend from the Divisions to the BBC’s Technology Group recognising that Siemens’s contracted margin is 12.25 per cent for commodity spend and 23 per cent for projects. The BBC anticipates that centralising control over project spend will: n provide it with a complete picture of the value of projects commissioned by the Division outside the Technology Framework Contract; n safeguard against project duplication; n lead to more successful, more productive projects; and n facilitate the spread of good practice across the BBC. Contractual value for money incentives 44 The prices set out in the Technology Framework Contract were arrived at following a competitive procurement process. Elsewhere, public sector organisations have kept prices competitive by introducing an overall profit-sharing provision, which is invoked if the private sector partner makes a return which exceeds a specified level. The contract with Siemens does not include a profit-sharing provision but includes the following provisions designed to keep prices competitive over the life of the deal. 7 Savings from increased flexibility under the contract with Siemens Under the contract with BBC Technology, the level of expenditure on desktop computers was fixed, irrespective of usage. Under the contract with Siemens, reduced usage results in lower charges. The BBC expected to make annual savings of £3.9  million from reduced demand for technology equipment in 2005-06, resulting from its redundancy programme. The redundancy programme was delayed while the BBC discussed the programme with its unions and the BBC now anticipates that the savings will be realised from 2006-07 onwards. Source: National Audit Office 9 An additional £11.6 million was paid to Siemens for ‘in-flight’ projects, adding up to the total spend under the contract of £190 million as at paragraph 41  above. 10 The Technology Service Advisors’ role is to reduce the volume of commodities purchased through the contract, help to ensure that appropriate equipment is purchased, use existing assets to their full potential and, where applicable, encourage Divisions to procure technology through the Siemens contract rather than from an alternative supplier. 45 Incentivised savings. Between years four and ten of the contract, Siemens is required to propose ways of achieving total net savings of £81.5 million over and above those savings already anticipated when the contract was signed. Siemens is required to show that the proposal would result in cost savings but if the BBC rejected a proposal because it decided it was unacceptable, Siemens would not be required to put forward an alternative. 46 Gain share. The gain share provision is designed to incentivise Siemens to identify and propose new projects over and above those proposed under the incentivised savings element of the contract. It does this by stipulating that any net savings resulting from gain share projects are split equally between Siemens and the BBC. 47 Open book accounting. The BBC has open book accounting rights that allow it to access and review financial information relating to the contract, including Siemens’s income, expenditure and profit from the deal. This gives the BBC the means to examine the rate of return to establish whether it is still in line with the level anticipated when the deal was competitively procured. The open book provision could, however, have been strengthened by providing the BBC with access to Siemens’s income and expenditure forecasts (as well as historical data). This would help the BBC to monitor anticipated returns and the sustainability of the deal. 48 The BBC has exercised its open book rights selectively to price some new projects but otherwise it has chosen to forego these rights for the first 18 months of the contract. It does not believe that there is a benefit to open book accounting during this period because it does not expect Siemens’s rate of return to be positive at this stage. As a result, the BBC does not know Siemens’s actual rate of return. 49 Price benchmarking. From the fourth year of the contract, the BBC can benchmark the price charged by Siemens for some services. Benchmarking could result in some cost savings but will not ensure that all prices stay competitive. The services which can be benchmarked represent just over two-thirds of estimated spend under the contract. The remaining one-third of services are not subject to benchmarking because Siemens and the BBC agreed that comparable benchmarks are not available. Prices would be compared to the average of the top 50  per cent of comparable service and prices could be up to 7.5  per  cent more expensive than this average before Siemens was required to reduce them. Are the BBC and Siemens taking the contract forward through partnership working? 50 The report by the Committee of Public Accounts on Managing the Relationship to Secure a Successful Partnership in PFI Projects highlighted the importance of developing a successful partnership in achieving value for money over the life of contracts.11 Although the contract between the BBC and Siemens is not a PFI contract, similar principles apply. “Co-location ensures close working and regular dialogue such that should any disputes arise, they can be resolved quickly” (Siemens comment) “There is good understanding [of the contract] at a senior and managerial level but this is not always shared at the more junior working or operational level” (BBC comment) Progress in establishing a partnering relationship 51 To encourage partnership working the BBC and Siemens teams are co-located, although there is still work to be done to establish the partnership at an operational level in BBC Divisions. In particular there is a need to improve the level of understanding of Siemens’s role among BBC Divisions. Both BBC and Siemens management recognise this and are developing plans to promote the partnership throughout the BBC. These include, for example, the new Board-level governance arrangements (paragraph 54), and promotion of the benefits that the partnership with Siemens can bring (Figure 8). 52 The scope for Siemens to win new business under the framework contract gives Siemens a financial incentive to develop the partnership. In the first year of the contract, Siemens was the exclusive supplier for around 77  per  cent of BBC spend, preferred supplier for 21 per cent and non -exclusive supplier for two per cent (paragraph 16). 11 Managing the Relationship to Secure a Successful Partnership in PFI Projects, Committee of Public Accounts Forty-Second Report, HC 460 Session 2001-02. 53 The contract includes a balanced scorecard which is also designed to provide a financial incentive to reward Siemens for high levels of partnership behaviour over and above routine service performance. The BBC can reward Siemens up to £0.5 million in any contract year, however, in practice the BBC and Siemens have been unable to establish an effective way of measuring high -level partnering performance. The scorecard has not been used since January 2006 and work is under way to develop a revised approach. 54 We consulted BBC and Siemens staff involved in overseeing and implementing the contract on how well the partnership is working. n Broadly, the people we consulted considered that: the roles and responsibilities between the BBC and Siemens are clear; there is a shared understanding of required outputs; and the BBC and Siemens share information freely and transparently. To support the contract, the BBC has drawn up a user guide to assist those people involved in the provision of services and in the relationship with Siemens. n The BBC’s Chief Technology Officer and his Siemens counterpart considered the relationship to be constructive. But the BBC and Siemens do not yet have the Board-level engagement necessary to drive the partnership forward. To this end, the partners are now setting up a new Technology Partnership Board on which will sit three BBC Board members – the Director-General, the Finance Director and the Director of New Media and Technology – and the BBC’s Chief Technology Officer. Siemens will be represented by its Executive Director responsible for the deal, the Managing Director and the Finance Director. The group will meet twice a year to agree strategic priorities for the contract and raise the profile of the contract inside the BBC. 55 Clarity in the management of risk is also likely to influence the success of the partnership. For shared risks we confirmed that the BBC and Siemens were producing a joint risk register, but one year into the contract this had not been finalised. The BBC has no separate register for the contract for those risks which rest with the BBC, although some risks are reported in its corporate risk register. Expanding the coverage of the contract 56 When the BBC’s technology requirements were provided by BBC Technology, parts of some BBC Divisions, in particular News, World Service, Radio and Music, and Nations and Regions, provided their own technology. These ’self-serve areas’ were not explicitly included under the original contract, although being a framework there is provision for them to benefit from it. The self-serve areas spend around £50 million a year on technology. From summer 2007, one part of the Nations and Regions Division, BBC Scotland’s IT and engineering support services, will be brought into the contract under call-off arrangements. In this case, the BBC estimated that the Siemens bid offered annual savings of £0.5 million calculated against a current spend of £28.1 million on the ‘in-house’ comparator. 57 The BBC considers that applying the contract to more of these self-serve areas has the potential to extend the anticipated benefits of the contract, and grow the partnership with Siemens to make the deal more sustainable. The BBC has not yet carried out a cost-benefit analysis of incorporating the self-serve areas into the contract to support this. 8 Partnership: Digital Fabric The Digital Fabric project aims, over a two-year period from October 2004, to deploy a new infrastructure which will allow different software applications to interact with each other, and provide the basis for the future rationalisation of applications across the BBC. Current projected costs are £6.2 million. The BBC expects the project to deliver cost savings by reducing the number of changes that need to be made to applications while Siemens anticipates that the project will provide opportunities to develop solutions which it could offer to customers other than the BBC. Siemens considers that the establishment of a strong partnership with the BBC allowed it to get the best deal when it went to the market with the BBC’s requirement. The BBC gave Siemens the freedom to use its expertise and knowledge to negotiate with vendors to get the best deal. Previously, the BBC had gone to the market for a similar requirement and negotiated directly with vendors, but had been unable to identify a suitable vendor. Source: National Audit Office Maintaining business continuity During the contractual term 58 Good practice guidance encourages organisations to plan effectively for possible early termination. The BBC has included safeguards within the contract designed to preserve business continuity and provide compensation if the partnership disintegrates. Some business critical services are ‘controlled’, meaning that the BBC has only released limited responsibility to Siemens. Siemens cannot move staff and assets from these controlled services to other assignments without BBC prior approval. In addition the BBC has secured the right to ‘step in’ and take over services for a period of time if a Siemens failure creates an immediate and serious threat to the BBC’s business or operational effectiveness. Pre-step in rights also exist which compel Siemens to investigate, report on remedial steps and provide the BBC access to Siemens’s information and individuals. 59 The Comptroller and Auditor General’s report on the National Savings and Investments’ deal with Siemens12 highlighted the importance of having a parent company guarantee to place the onus on the supplier to take remedial action in the case of financial difficulty. In line with this, the BBC’s contract with Siemens includes a parent company guarantee to provide a disincentive for Siemens to pull out of the partnership before the end of the contract term. 60 The Comptroller and Auditor General’s report on the National Savings and Investments’ deal with Siemens13 also recommended that organisations should have contingency plans in place in case private sector partners default, and that such plans should be drawn up and maintained even where default is unlikely. For the first year of its contract with Siemens, the BBC did not have contingency plans in the event of contract termination, but these plans are now being developed. At the end of the contract 61 Reports by the Committee of Public Accounts on the Inland Revenue/EDS Strategic Partnership14 and the NIRS2 Contract Extension15 concluded that it could be difficult to create effective competition for a large contract with a strong incumbent supplier. Good practice guidance on outsourcing16 sets out what an organisation can do to promote effective competition when a contract expires and support a smooth transfer of services to a successor provider. 62 In line with this good practice, and drawing on lessons learned from other BBC outsourcing deals, the BBC’s contract with Siemens includes provisions to support re-competition and the transfer of services. Most importantly, Siemens’s responsibilities are set out in an exit plan; ownership and access rights to tangible property and intellectual property are made clear; arrangements are in place to transfer key staff; and the contract has information-sharing provisions designed to level the playing field between Siemens and prospective bidders. These contract provisions should help the BBC to manage the end of the contract but success will also depend on the BBC’s ability to promote competitive tension in the market place and manage the relationship between Siemens and other bidders. 12 PPP in practice: National Savings and Investments’ deal with Siemens Business Services, four years on, HC 626, Parliamentary Session 2002-2003. 13 PPP in practice: National Savings and Investments’ deal with Siemens Business Services, four years on, HC 626, Parliamentary Session 2002-2003. 14 Inland Revenue/EDS Strategic Partnership: The Award of New Work (28th report 1999-2000). 15 NIRS2 Contract Extension (38th report 2001-2002). 16 For example, the Office of Government Commerce successful delivery toolkit. The cont ra ct between the BBC and Siemens Business Se rvi ces fo r the p rovision of te chnolo gy se rvi ces 1 9 Apendix 1 Methodology Aim and scope The BBC Governors asked the National Audit Office to examine whether the BBC is well placed to deliver the anticipated benefits over its 10-year life. We did this by assessing whether the BBC is equipped to realise the anticipated benefits of the contract in the future, and early performance against the BBC’s objectives (while recognising that the contract is at an early stage). We did not seek to address the value for money of the sale of BBC Technology Ltd or the procurement of Siemens, although to understand whether the contract will meet the BBC’s objectives it was necessary to reflect on how the deal was negotiated. Methods Documentary review and analysis We reviewed BBC documentation and data relevant to the Siemens deal, including the Technology Framework Contract and supporting schedules, business case and approval papers, service level and financial performance data, project delivery reports, contract management process documentation and Internal Audit reviews. We also reviewed documentation relating to other BBC outsourcing deals to consider whether lessons learnt could be applied to technology outsourcing. Consultation We interviewed BBC senior management and other key staff involved in setting up and managing the Technology Framework Contract. This included staff from the BBC’s technology, finance and legal teams, who provided us with information on early performance and the arrangements in place to secure value for money over the contract’s 10-year life. We also interviewed representatives from Siemens Business Services to obtain their perspective on the contract and the ongoing partnership with the BBC. We liaised closely with the BBC’s Business Assurance team and met staff from other BBC outsourcing contracts to understand the wider context of BBC outsourcing and discuss lessons learnt from other deals. Drawing on good practice guidance from previous National Audit Office reports and the Office of Government Commerce successful delivery toolkit,17 we developed a set of questions to examine the effectiveness of partnership arrangements to date and obtain views on where the relationships could be strengthened. The questions were targeted at a small sample of key BBC and Siemens staff selected from both management and staff working on a sample of technology projects. Review of other deals Our work was informed by lessons learnt from National Audit Office reviews of large public sector outsourcing deals and other good practice guidance. Our sources are outlined in Appendix 2. 17 Sources provided in Appendix 2. The cont ra ct between the BBC and Siemens Business Se rvi ces fo r the p rovision of te chnolo gy se rvi ces 20 Apendix 2 Good practice references In producing this report, we were guided by good practice principles from the following sources: The effective design and use of the performance measurement systems n Choosing the right FABRIC – a framework for performance information (joint publication by the National Audit Office, HM Treasury, Cabinet Office, Audit Commission and Office for National Statistics). Managing and controlling contractual costs n The Office of Government Commerce successful delivery toolkit: module 10 on financial issues. Contractual mechanisms to maintain and improve value for money by promoting high service levels and reducing costs n PPP in practice: National Savings and Investments’ deal with Siemens Business Services, four years on: HC 626, Session 2002-2003. n Delivering better value for money from the Private Finance Initiative: Committee of Public Accounts, 28th report, HC 764, Session 2002-03. n The Office of Government Commerce successful delivery toolkit: module 10 on financial issues and model IT services agreement – Annex to Part 4. Developing a strong partnering relationship to help achieve value for money over the lifetime of contracts n Managing the relationship to secure a successful partnership in PFI projects: Committee of Public Accounts 42nd report, HC 460, Session 2001-02. n The Office of Government Commerce successful delivery toolkit: effective partnering. Exit management and business continuity planning to maintain business as usual at the end of the contract or in the event of early termination n PPP in practice: National Savings and Investments’ deal with Siemens Business Services, four years on: HC 626, Session 2002-2003. n The Office of Government Commerce successful delivery toolkit: module 8 on exit management. Supporting a future re-tendering to encourage competition when the contract is re-let n The Office of Government Commerce successful delivery toolkit: recompetition. n Inland Revenue/EDS strategic partnership – the award of new work: Committee of Public Accounts, 28th report, HC 431, Session 1999-2000. n NIRS2 contract extension: Committee of Public Accounts, 38th report, HC 423, Session 2001-02. n The Office of Government Commerce successful delivery toolkit: Inland Revenue case study. Apendix 3 Key service targets under the contract Key service target Year one target Customer satisfaction – overall service (survey of BBC staff) 50% Changes under Siemens’s control having an unplanned impact on service levels 1.55% Percentage of projects delivered within governance framework 90% Project deliverables completed on time 90% Breaches of occupational risk 0 Availability of internet – internal measure 99.95% Availability of internet – external measure 99.5% Availability of internet – content acquisition and distribution 99.54% Availability of radio – external measure 99% Availability of radio – internal measure 99.5% Availability of analogue TV distribution 100% Availability of digital satellite television – UK 100% Availability of digital satellite television – Nations and Regions 100% Availability of digital terrestrial television – UK 100% Availability of digital terrestrial television – London 100% Availability of sustaining television feeds – Nations and Regions 100% Availability of clean television feeds – Nations and Regions 100% Availability of digital satellite television – uplinks 100% Availability of analogue terrestrial television distribution (annual) 99.998% Availability of digital terrestrial television distribution (annual) 99.998% Availability of UK and Nations analogue terrestrial radio (annual) 99.998% Number of detected external security breaches 0 Number of severe gaps in compliance with ISO17799 – best practice on information security 2 management (annual) Customer satisfaction – ‘call-off’ project support (survey of BBC staff) 50% Availability of the internal telephone service 99.95% Resolution times for ‘telephony critical’ faults 97% Resolution times for ‘broadcast critical’ incidents (within 20 minutes) 90% Resolution times for ‘critical’ incidents (within 1 hour) 90% Resolution times for ‘normal’ incidents (within 8 hours) 95%