Public service broadcasting: the BBC’s 
performance measurement framework 
NAO review presented to the BBC Governors’ Audit Committee, 
May 2005, by the Comptroller and Auditor General, and a response 
to the review from the BBC 


In Building public value the BBC committed 
itself to introducing a new system of 
performance monitoring based on 
objectivity, rigour and transparency. 
The objective was to strengthen the 
Board of Governors’ ability, on behalf of 
licence fee payers, to ensure that BBC 
services deliver against their wider public 
purpose.The performance measurement 
framework is the first of a number of 
related initiatives being introduced as part 
of this commitment. 
This external Value for Money review, 
commissioned by the Governors’ Audit 
Committee, considered public service 
broadcasting in terms of the BBC’s 
performance measurement framework. 
It was undertaken by the National Audit 
Office, following the agreement in 2003 
between the Government and the BBC 
to evolve the BBC Governors’ overview 
of value for money into a programme of 
reviews.This review focused on the key 
drivers of public value – reach, quality, 
impact and value for money. 
Governors welcome the independent 
review’s confirmation of the framework 
as a rigorous and appropriate method to 
assess the delivery of public value, with the 
BBC appearing to be at the “forefront of 
current thinking on the topic”. However, 
we recognise there is further work to be 
done and endorse BBC management’s 
response to the report’s recommendations. 
The NAO report highlights evidence of 
the framework beginning to “influence 
thinking and decision-making about public 
service broadcasting within the BBC”.This 
goes to the heart of the framework’s 
purpose, fully aligning the BBC’s decisionmaking 
with the public interest. Governors 
believe that the framework provides both 
a consistent and coherent framework 
within which decisions can be made and 
measures against which the BBC’s delivery 
of public value can be assessed.We 
support the NAO’s recommendation that 
performance reports should be tailored to 
the needs of different audiences. Governors 
will work with management both to 
facilitate assessment of key performance 
indicators and, as custodians of the public 
interest in the BBC, to ensure the delivery 
of public value to licence fee payers. 
Governors accept that there is a need to 
reflect the performance framework in the 
BBC’s overall objectives for public service 
broadcasting and that the link between 
performance monitoring and the 
Corporation’s public service broadcasting 
objectives can be made more explicit.The 
reformed governance system, as set out in 
the Green Paper, has implications for the 
system and structure of performance 
monitoring, which will in turn influence the 
ways in which the proposed BBC Trust 
accounts for the BBC’s performance 
against key measures in the Annual Report 
and to Parliament.The details of these 
changes are currently being worked out 
and Governors will outline their plans in 
due course. 
In summary, the Board of Governors is 
pleased to note the NAO’s conclusion 
that “The BBC has made good progress in 
developing the performance measurement 
framework [with] important building 
blocks in place…”We have considered 
and approved the attached response from 
BBC management. 
Board of Governors 
June 2005 
Public service broadcasting: the BBC’s 
performance measurement framework 
BBC response to the National Audit Office Value for Money study

The performance measurement 
framework, introduced in autumn 2004, is 
evidence-based and designed to provide 
BBC Governors and management, licence 
fee payers and other stakeholders with 
assurance that BBC services are delivering 
public value. 
The key focus of the framework reviewed 
by the NAO is on the four drivers of 
public value – reach, quality, impact and 
value for money.The NAO has recognised 
that these drivers share much common 
ground with the characteristics of public 
service broadcasting set out by Ofcom in 
its review of public service broadcasting. 
Response to the report’s 
key findings 
The NAO has acknowledged within its 
report the BBC’s achievements to date 
in introducing the framework. In particular 
it notes that the new performance 
measurement framework arrangements 
have the potential to work for the BBC 
on two levels. In the broadest terms, the 
drivers of reach, quality, impact and value 
for money give the BBC a common 
framework and language for thinking about 
public service broadcasting. At a detailed 
level, the technical measurement of 
performance against these four drivers 
provides an evidence base for decisionmaking, 
performance monitoring 
and accountability. 
The study also recognised that the BBC’s 
drivers of public value is consistent with 
the Government’s Green Paper on the 
review of the BBC’s Royal Charter. In 
addition, the BBC’s continuing commitment 
to developing the framework is reflected 
through the acknowledgment that the 
BBC appears to be at the forefront of 
current thinking on the topic and 
continues to liaise with other public 
service broadcasters in Europe 
and beyond. 
The NAO does, however, also note that 
there is further work required in 
developing and refining the current system, 
which the BBC should address over the 
next 12 months.The BBC therefore 
welcomes the input of the NAO into the 
framework at this stage of development 
and sets out below how these 
recommendations will be progressed. 
Responding to the 
recommendations 
and conclusions 
Since the review fieldwork between 
January and March 2005, the BBC has 
undertaken a number of actions. 
•The Quarter 1 2005 performance 
report has been published, with a more 
user-friendly opening section (Talking 
Points) which highlights the key issues 
for the period. 
• Detailed objectives have been set for all 
divisions in the BBC, with clear targets 
agreed by the Director-General. Output 
commissioning divisions have been set, 
Reach, Quality, Impact and Volume 
(RQIV) targets (for example, increase 
reach by X%, improve ‘channel best for’ 
scores from X to Y.) Output divisions 
without commissioning responsibilities 
have been given relevant quality and 
value for money targets.These will 
be monitored via the quarterly 
reporting process. 
• An internal reporting group has been 
set up to look at more effective ways 
of reporting financial indicators within 
the RQIV framework. 
The BBC welcomes the NAO’s input into 
the continuing development and 
refinement of the performance 
measurement framework.The report 
draws attention to the following areas 
where further work could be undertaken 
by the BBC over the next 12 months. 
• Establish clear definitions of reach, quality, 
impact and value for money and the 
supporting performance measures to help 
achieve consistency across the organisation. 
The BBC accepts this recommendation 
and aims to integrate the formal 
divisional targets and supporting 
measures into the reporting framework 
over the summer. Once this framework 
has been produced a performance 
measurement ‘manual’ will be produced 
for users across the organisation, to be 
implemented by the September 2005 
quarterly performance report. 
• Assess the extent to which the 
performance measurement framework 
is embedded at all levels of the BBC. 
The BBC recognises that the 
framework may take some time to 
become embedded throughout the 
whole organisation and proposes to 
conduct regular reviews with the 
relevant divisions in order to identify 
any issues and recommend ways of 
embedding the framework more 
effectively.These reviews will be carried 
out at least once a year. 
• Ensure that there is an explicit focus on 
providing assurance about the accuracy 
and reliability of the performance 
measurement framework. 
The BBC is confident that appropriate 
assurance provisions are in place with 
its main research suppliers (BARB, 
RAJAR and TNS). All performance data 
provided for external publication, for 
example in the BBC’s Annual Report 
and Accounts, are currently subject to 
an internal assurance process. However, 
the BBC recognises the lack of a formal 
assurance programme across the whole 
performance reporting process to date. 
The BBC’s internal audit function will 
undertake an operational review of 
performance reporting by the end of 
Public service broadcasting: the BBC’s 
performance measurement framework 
BBC management response

2005, when the new system will have 
been through a full year’s reporting 
cycle.This review is likely to be 
repeated on an annual basis. 
• Make the performance reports more userfriendly, 
principally by tailoring the reports 
to meet the needs of different audiences 
and by differentiating between priority 
and less important information. 
Different versions of the reports are 
already being produced for board level 
and general staff consumption.We will 
closely monitor user satisfaction of the 
reports as part of the ongoing review 
process. Individual performance targets 
have been agreed with BBC output 
divisions for 2005/2006 and these will 
be highlighted in the September report 
to draw the distinction between formally 
agreed targets and other measures. 
In addition to the points raised above, two 
further areas for the BBC to address are 
highlighted by the NAO.These are 
addressed in the Governors’ response. 
• Reflect reach, quality, impact and value for 
money in the BBC’s overall objectives for 
public service broadcasting. 
• Set a timetable for reporting on reach, 
quality, impact and value for money to 
Parliament and the public.


Public service broadcasting: the BBC’s 
performance measurement framework 
REVIEW BY THE COMPTROLLER AND AUDITOR GENERAL 
PRESENTED TO THE BBC GOVERNORS’ AUDIT COMMITTEE | 27 May 2005 

T his report has been prepared under 
Clause 10 B of the amendment to the 
A greement between the S ecretar y of S tate 
for Culture , Media and S port and the BBC, 
dated 4 D ecember 2003. 
John Bourn 
Comptroller and Auditor General 
National Audit Office 
T he N ational A udit O f fi ce 
stud y team consisted of: 
L aura Brackwell, K eith H awkswell, 
J onathan Macka y and A ilbhe Mc N abola 
A link to this report can be found on the 
N ational A udit O f fi ce web site at 
www. nao . org . uk 
F or furth er information a b out th e 
National Audit Office plea s e contact : 
N ational A udit O f fi ce 
Press O f fi ce 
1 57- 1 97 Buckingham Palace R oad 
V ictoria 
L ondon 
S W 1 W9 S P 
T el: 020 7798 7 400 
E mail: enq uiries @ nao . gsi . gov . uk 
© N ational A udit O f fi ce 2005 
CONTENTS 
EXECUTIVE SUMMARY 1 
PART 1 
I ntroduction to the performance 4 
measurement framework 
W here does the performance measurement 5 
framework sit within the BBC? 
H ow do the BBC’s drivers of public value link 6 
with current thinking on the characteristics of 
public service broadcasting? 
H ow does the BBC’s approach to measuring 7 
public service broadcasting compare with overseas ? 

PART 2 
T he design of the performance 8 
measurement framework 
H as the BBC identi fi ed performance measures for 9 
the four drivers of public value ? 
D oes the performance measurement framework 12 
focus on the BBC’s ob j ectives ? 
D oes the performance measurement framework 12 
cover all the BBC’s public service broadcasting? 
D o the performance measures allow the BBC to 13 
benchmark its performance with others? 
PART 3 
T he data sy stems supporting the 14 
performance measurement framework 
A re sy stems in place to generate comprehensive 1 5 
and timely data ? 
D oes the BBC have adeq uate q ualit y assurance 1 7 
arrangements? 
PART 4 
H ow performance information is being 1 8 
used and reported 
I s the performance measurement framework being 1 9 
used to infl uence thinking in the BBC? 
A re the arrangements for reporting 20 
performance effective ? 
D oes the performance measurement framework 20 
provide a basis for the BBC to be e x ternally 
accountable? 
APPEND IX 
S tud y methods 22 
Photographs courtes y of the BBC 

e x ecutive summar y 
EXECUTIVE SUMMARY 

e x ecutive summar y 
PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 
1 T his report is about the new performance 
measurement framework which the BBC began to introduce 
in A utumn 2004. T he framework is a new evidence - based 
s y stem of performance measurement designed to give 
BBC G overnors and Management , licence fee pa y ers 
and other stakeholders assurance that BBC services are 
delivering public value . T he framework is part of the 
arrangements set out in ‘ Building Public V alue’ , which the 
BBC published in J une 2004 to contribute to the debate on 
the review of its R o y al Charter which e x pires in 2006 . 
2 A t the heart of the performance measurement 
framework are four key drivers of public value identified 
b y the BBC – re a ch, qua lit y , impa ct and v a lue for 
money . T hese drivers share much common ground with 
the characteristics of public service broadcasting set 
out b y O fcom1 in its review of public service television 
broadcasting. T he BBC’s drivers are also consistent with 
the G overnment’s recent G reen Paper on the R eview 
of the BBC’s R o y al Charter . T he BBC’s framework is 
similar to approaches being developed b y public service 
broadcasters overseas and the BBC appears to be at the 
forefront of current thinking on this topic . 
3 T he BBC is continuing to develop and refine the 
framework and to an e x tent is learning as it goes . S o our 
aim has been to review progress and contribute to the 
BBC’s thinking on the further work to be done. I n doing 
our work , we were guided b y the good practice principles 
set out in the publication ‘ Choosing the right F A B RIC - a 
framework for performance information’ 2 , which identifies 
the key components of a good performance measurement 
s y stem ( set out in the A ppendix to this report) . A gainst this 
background, we considered: 
p the design of the performance measurement 
framework ; 
p the data sy stems supporting the performance 
measurement framework ; 
p how performance information is used and reported. 
4 O ur work was not about the BBC’s editorial and 
programming j udgements or whether BBC services fulfil 
its public service broadcasting remit , but was about the 
design and implementation of the new framework . A s it is 
still in its earl y da y s , we considered it too soon to conduct 
a detailed test of how well the BBC matches up to the 
‘F A B RIC’ guidance . I nstead we were concerned with the 
general direction the BBC is going in with the framework . 
O ur approach to the e x ercise is outlined in the A ppendix . 
O ur main findings 
5 T he BBC has made good progress in developing 
the performance measurement framework . I mportant 
building blocks are in place in that the BBC has identified 
performance measures , data collection is underwa y, and 
performance is being reported to S enior Management 
and the Board of G overnors. T here is also evidence of the 
framework beginning to influence thinking and decision 
making about public service broadcasting within the BBC. 
1 O fcom ( the O ffice of Communications ) was established b y the Communications A ct 2003 as the independent regulator and competition authorit y for the 
U nited K ingdom communications industries , with responsibilities across television, radio, telecommunications and wireless communications services . 
2 Choosing the right F A B RIC – a framework for performance information ( H MTreasury, Cabinet O ffice , N ational A udit O ffice , A udit Commission and O ffice for 
N ational S tatistics ) . 

e x ecutive summar y 
PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 2 
6 O n the design of the perform a nce mea surement 
fr a mework, the BBC has identified a package of measures 
which will support its assessment of performance for 
each of the four drivers of public value – reach , q ualit y, 
impact and value for money. H owever , at present there are 
instances of different BBC divisions using the same measure 
to assess different drivers – formalising definitions for use 
across the organisation would help achieve consistenc y. 
7 I n designing the performance measurement 
framework , the BBC has identified a range of q uantitative 
measures, which give it a clear basis for tracking 
performance over time and benchmarking against 
other broadcasters. H owever , man y of the measures 
are necessarily perception- based and, as the BBC 
recognises, the results in these areas tend to re q uire more 
interpretation than in others where the data is harder . 
8 T o get the most out of the performance measurement 
framework , there should be a direct link between the 
framework and the BBC’s ob j ectives for the deliver y of 
public service broadcasting. T he BBC is in the process of 
agreeing ob j ectives for individual broadcasting divisions in 
reach , q ualit y, impact and value for money terms but has 
not linked ob j ectives for the organisation as a whole to the 
performance measurement framework . T he framework is 
being applied in full in si x of the eight BBC broadcasting 
divisions funded b y the licence fee – the two e x ceptions are 
divisions which supply programming to the other divisions 
and have less influence over the performance drivers. 
9 O n the da t a s y stems supporting the perform a nce 
mea surement fr a mework, the sy stems the BBC has in 
place are generating data for each of the four performance 
drivers. S ome of the sy stems are longstanding, using 
well established industry sources such as B AR B ( the 
Broadcasters’ Audience R esearch Board ); and the BBC has 
introduced others to improve the q ualit y and timeliness 
of the data , including a new I nternet surve y designed to 
generate richer data on q ualit y and impact . At present 
the BBC’s q ualit y assurance arrangements do not focus 
e x plicitl y on the performance measurement framework . 
1 0 O n the use a nd reporting of inform a tion gener a ted 
by the perform a nce mea surement fr a mework, the 
performance drivers – reach , q ualit y, impact and value 
for money – are becoming common currenc y at senior 
levels in the BBC and work to embed the framework 
throughout the organisation is continuing. Performance 
reports are produced and considered q uarterl y at Board 
level but there is scope to make them sharper and 
more user - friendly, and to differentiate between priorit y 
information and material that is of secondar y importance . 
T he reports are also not tailored to different audiences – at 
present the Board of G overnors and S enior Management 
Boards receive the same report. 
11 As regards e x ternal reporting, ‘ Building Public 
Value’ stated the G overnors’ intention to use the measures 
of reach , q ualit y, impact and value for money to set 
ob j ectives for the BBC and its services each y ear , and that 
they would form the basis for the BBC’s Annual R eport to 
Parliament and the British public . 

e x ecutive summar y 
PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 3 
12 T he new performance measurement arrangements have the potential to 
work for the BBC on two levels. I n the broadest terms, the drivers of reach , 
q ualit y, impact and value for money give the BBC a common framework 
and language for thinking about public service broadcasting. At a detailed 
level, the technical measurement of performance against these four drivers 
provides an evidence base for decision making, performance monitoring 
and accountabilit y. 
13 T he BBC is continuing to refine the performance measurement framework 
and the main te x t of this report identifies a number of areas where there is 
further work to be done. O ver the nex t 12 months, it will be particularl y 
important for the BBC to: 
p reflect reach , q ualit y, impact and value for money in the BBC’s overall 
ob j ectives for public service broadcasting; 
p establish clear definitions of reach , q ualit y, impact and value for money 
and the supporting performance measures to help achieve consistenc y 
across the organisation; 
p assess the e x tent to which the performance measurement framework is 
embedded at all levels of the BBC; 
p ensure that there is an e x plicit focus on providing assurance about the 
accuracy and reliabilit y of the performance measurement framework ; 
p make the performance reports more user - friendly, principally b y tailoring 
the reports to meet the needs of different users and b y differentiating 
between priorit y and less important information; 
p set a timetable for reporting on reach , q ualit y, impact and value for 
money to Parliament and the public . 
OUR CONCL UD ING COMMENTS AND RECOMMEND ATIONS 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part one 
4 
PART ONE 
I ntroduction to the performance measurement framework 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part one 
5 
W here does the performance 
measurement framework sit within 
the BBC? 
1 . 1 T he British Broadcasting Corporation ( BBC) is the 
U nited K ingdom’s main public service broadcaster , 
receiving licence fee revenue ( £ 2. 8 billion in 2003- 04 ) 
and free access to the spectrum in return for delivering 
public service broadcasting. F i g ure 1 provides information 
on the BBC’s organisational structure . 
1 . 2 U nder its R o y al Charter , the BBC’s ob j ects include 
“ to provide, as public services , sound and television 
broadcasting services … and to provide sound and 
television programmes of information, education and 
entertainment for general reception…” . T o meet this 
re q uirement , across the U nited K ingdom the BBC 
currentl y provides: 
p eight television channels – two analogue ( BBC O ne 
and BBC T wo ) and si x digital ( BBC T hree, BBC F our, 
CBBC, CBeebies , BBC N ews 24 and BBC Parliament ); 
p ten radio networks – five analogue ( BBC R adios 1, 
2, 3, 4 and F ive L ive ) and five digital ( BBC F ive L ive 
S ports E x tra , 1X tra , BBC 6 Music , BBC 7 and BBC 
A sian N etwork ) – and local and national radio stations ; 
p new media – the online site , bbc. co . uk , and BBCi , 
the BBC’s interactive services . 
I n 2003- 04 the BBC’s total spending on broadcasting 
( including overheads ) was £ 3.0 billion. 
1 The BBC’s organisational structure 
p The BBC is a public corporation, established in 1927 by 
a renewable Royal Charter. Many of the BBC’s public 
obligations are set out in an accompanying Agreement 
with the Secretary of State for Culture, Media and Sport. 
p The BBC is overseen by a Board of 12 Governors, 
appointed by The Queen on advice from Ministers. 
The Board of Governors is responsible for ensuring that 
the BBC meets its statutory and other obligations and 
that it is managed in the public interest by, among other 
things, approving strategy and policy, setting objectives 
and monitoring performance. The Board of Governors 
is supported in its work by the recently established 
Governance Unit. 
p The Board of Governors appoints the BBC’s Director- 
General and the eight other members of the Executive 
Board, which is responsible for managing the BBC. 
The Executive Board is supported by three boards covering 
the BBC’s main activities – the Creative Board (also chaired 
by the Director-General), the Journalism Board and the 
Commercial Board. 
p The BBC’s day to day operations are run by 16 divisions 
– nine broadcasting divisions which manage the BBC 
networks and their programme commissioning and 
production; five divisions which provide professional 
support; and two commercial divisions which sell goods 
and services around the world. 
Source: National Audit Office 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part one 
6 
1 . 3 T he BBC’s current R o y al Charter e x pires on 
31 D ecember 2006 and is presentl y being reviewed b y the 
D epartment for Culture , Media and S port. T o contribute to 
the debate on the Charter , in J une 2004 the BBC published 
‘ B uilding P ublic Value – renewing the BBC for a digital 
world’ , which describes how the BBC adds value to life in 
the U nited K ingdom and sets out how it intends to build 
on this in the future . T he document sets out five wa y s 
the BBC creates public value , which it terms its public 
purposes ( F i g ure 2 ) . T he G overnment took account of the 
BBC’s views in developing the public purposes for the 
BBC proposed in the G reen Paper on the R eview of the 
BBC’s R o y al Charter , published in March 2005 . 
1 . 4 ‘ Building Public V alue’ also outlines the BBC’s plans 
for a new sy stem designed to demonstrate its deliver y of 
public value , comprising four components for assessing 
new services and monitoring e x isting services ( F i g ure 3 ) . 
1 . 5 I n addition, under the Communications A ct 2003 
the BBC along with other public service broadcasters is 
re q uired to publish a ‘ S tatement of Programme Polic y ’ , 
describing how in the coming y ear it will fulfil its public 
service remit and reporting performance against the 
previous y ear’s proposals . T he S tatements set out minimum 
re q uirements for programming in areas such as news 
and current affairs, independent production, regional 
programming, and access to services for deaf 
and visually - impaired audiences . 
H ow do the BBC’s drivers of 
public value link with current 
thinking on the characteristics 
of public service broadcasting? 
1 . 6 U nder the Communications A ct 2003, O fcom is 
re q uired to carry out ever y five y ears a review of public 
service broadcasting television. O fcom carried out the 
first of these reviews in three phases , culminating in the 
publication of its final report in F ebruar y 2005 . 
1 . 7 A s part of its review , O fcom defined the 
characteristics of public service broadcasting. Comparing 
O fcom’s characteristics with the BBC’s drivers of public 
value on which the performance measurement framework 
is based indicates that , while e x pressed differentl y, they 
share much common ground although in its definition 
the BBC has the additional driver of value for money 
( F i g ure 4 ) . O fcom told us that it considered the BBC’s 
drivers of public value were ver y much in line with its 
own characteristics , reflecting the e x tensive dialogue 
between the two bodies . G oing forward , it will be 
important for the BBC to continue to liaise with O fcom 
on these matters. 
1 . 8 T he BBC’s drivers of public service broadcasting are 
also consistent with the G overnment’s recent G reen Paper 
on the R eview of the BBC’s R o y al Charter , which proposed 
that the BBC should provide a wide range of content , 
across ever y programme genre , try ing to reach the 
greatest possible range of audiences . Programmes should 
aim to be e x cellent , distinctive and entertaining – more 
specifically they should be of high q ualit y, challenging, 
original , innovative and engaging. A s the Charter review 
process goes forward , it will be important for the BBC to 
ensure that its performance measurement framework takes 
account of this wider conte x t . 
2 The BBC’s public purposes – how it creates 
public value 
Source: ‘B uilding P u b lic V alue ’ (BBC, J une 2004) 
p Democratic value: supporting informed citizenship 
p Cultural and creative value: enriching the creative life of the 
United Kingdom 
p Educational value: extending horizons 
p Social value: connecting and uniting communities 
p Global value: supporting the United Kingdom’s role 
in the world 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part one 
7 
H ow does the BBC’s approach 
to measuring public service 
broadcasting compare with overseas ? 
1 . 9 W e looked at how public service broadcasters 
in other countries were approaching performance 
measurement . O ur overall sense was that the BBC is at the 
forefront of current thinking on this topic and continues to 
liaise with other public service broadcasters in E urope and 
be y ond. A nd the E uropean Broadcasting U nion3 , of which 
the BBC is a member , is currentl y considering approaches 
to the measurement of performance in public service 
broadcasting. 
1 . 1 0 Points of particular interest are noted at relevant 
points in this report but in general terms other public 
service broadcasters, in particular in S candinavian 
countries , are implementing performance measurement 
structures similar to those being implemented b y the BBC. 
T he public service broadcaster in the N etherlands ( Publieke 
O mroep) uses a performance ‘ q ualit y card’ , comprising 
measures of reach and share , q ualit y, impact and efficienc y. 
3 The BBC’s proposals for assessing and monitoring public value 
Source: National Audit Office , dra w ing on ‘B uilding P u b lic V alue ’ (BBC, J une 2004) 
Public value test 
Service licences 
Public value survey 
P erformance measurement framework 
The public value test will be used to judge the contribution a service would potentially make 
to the delivery of public value by the BBC. The test will include an independent evaluation 
of a service’s market impact and audience research, and will be applied to all new service 
proposals and significant changes to existing services. 
In March 2005 the Board of Governors commissioned Spectrum Strategy Consultants to 
assess whether the public value test is robust enough to be the main tool used in deciding 
whether to invest public money in markets in which there may be commercial players. 
Service licences are to be granted to each BBC channel and service by the Board of 
Governors. A licence will set out a service’s remit, purpose and contribution to public value, 
including performance targets. 
The licences are currently being developed and the BBC expects to introduce them in a 
shadow form in 2006 and in full in 2007. The licences will be published. 
Every three to five years, the Board of Governors will commission an independent public 
value survey of 10,000 licence fee payers. The survey will be designed to provide a 
detailed, audience-based assessment of the impact, value and effectiveness of the BBC’s 
services. The results will be published. 
The performance measurement framework is a new evidence-based system of performance 
measurement which aims to give BBC Governors, licence fee payers and other stakeholders 
greater assurance that existing BBC services are delivering public value. 
T his report focuses on the performance measurement framework which is centred on four key 
drivers of public value identified by the BBC – reach, quality, impact and value for money. 
4 Comparing the BBC’s drivers and Ofcom’s 
characteristics of public service broadcasting 
T he BBC ’s drivers 
R each – providing universal 
availability and access 
Q uality – providing high quality, 
distinctive programming 
I mpact – enriching the lives of 
individuals and citizens 
V alue for money – providing the best 
value for money for licence fee payers 
O fcom’s characteristics 
Widely available 
High quality 
Original 
Innovative 
Challenging 
Engaging 
Source: National Audit Office 
3 T he E uropean Broadcasting U nion is a professional association of national broadcasters which , among other things , negotiates broadcasting rights, provides 
services and represents the interests of public service broadcasters. 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
8 
PART T W O 
T he design of the performance measurement framework 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
9 
2 . 1 T his part of the report considers the design of 
the BBC’s new performance measurement framework . 
I nformed b y the ‘ Choosing the right F A B RIC’ guidance 
( see paragraph 3 of the E x ecutive S ummar y ) , we focused 
on a number of high- level q uestions which will be key as 
the BBC takes the new framework forward: 
p whether the BBC has identified performance 
measures for the four drivers of public value ; 
p whether the performance measurement framework 
focuses on the BBC’s ob j ectives ; 
p whether the performance measurement framework 
covers all the BBC’s public service broadcasting; 
p whether the performance measures allow the BBC 
to benchmark its performance with others. 
2 . 2 T he embodiment of the performance measurement 
framework is the information presented in the BBC’s 
performance reports. S o our anal y sis focused on the 
information set out in the summar y reports, which cover 
the performance of the BBC as a whole; and we also 
looked at the reports produced at divisional level, which 
feed into and supplement the summar y reports. 
H as the BBC identified performance 
measures for the four drivers of 
public value ? 
2 . 3 T he BBC has identified measures of performance for 
each of the four drivers in the performance measurement 
framework – reach , q ualit y, impact and value for money. 
S ome of these have been carried forward from previous 
performance sy stems and others are new measures 
developed specifically for the framework . 
2 . 4 T he performance measurement framework is still 
relatively new and developing, and going forward it will 
be important for the BBC to: 
p establish clear definitions of reach , q ualit y, impact 
and value for money, and the supporting measures . 
T he BBC staff most heavily involved in the framework 
appear to have broad working definitions but 
formalised definitions approved for use across 
the organisation would help achieve consistenc y. 
A t present there are instances of different BBC 
divisions using the same measure to assess different 
performance drivers. F or e x ample, approval ratings are 
used as a measure of q ualit y b y the T elevision division 
and as a measure of impact b y several other divisions . 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
1 0 
p maintain a balance between reach , q ualit y, impact 
and value for money so that no single performance 
driver has an undue influence . A strength of 
the framework is that the package of drivers 
come together to provide an overall picture of 
performance . But focusing on an y individual driver 
risks unbalancing the picture as ‘ good’ results in one 
area ma y be achieved at the e x pense of ‘ bad’ results 
in another . F or e x ample, high q ualit y programming 
could be more e x pensive and increase the cost per 
viewer hour, and vice versa . 
( a ) R each 
2 . 5 R each is the headline indicator of take- up of the 
BBC’s services and has the benefit of lending itself to 
firmly q uantifiable measures . T he wa y s the BBC currentl y 
measures reach are summarised in F i g ure 5 . 
2 . 6 A udience sh a re ( the percentage of the total audience 
watching or listening to a particular channel or service 
rather than to an y of the others over a given period of 
time) together with audience volumes has traditionally 
been the key measure of performance in broadcasting. 
A nd share continues to be part of the BBC’s new 
performance measurement framework . 
2 . 7 H owever , in line with the commitment it made in 
‘ Building Public V alue’ ( see paragraph 1.3 ) , the BBC now 
places greater emphasis on overall weekly re a ch – the 
percentage of the population who have viewed, listened to 
or used a BBC service for a consecutive period of 1 5 minutes 
over the course of a week. Placing greater emphasis on 
weekly reach reflects the BBC’s aim to serve the whole 
population whose licence fees help to fund its services . 
2 . 8 Clearl y though, reach and share are interrelated 
and an increase in one ma y result in an increase in the 
other . I n addition, audience share and volumes feed into 
a number of other performance measures , including the 
cost per viewer hour, one of the key measures of value 
for money ( see F igure 8 ) . T he BBC needs to ensure that 
these interrelationships do not distort the balance of the 
performance measurement framework – for e x ample, a 
particular emphasis on reducing cost per viewer hour 
could have the effect of prioritising audience share over 
weekly reach . 
2 . 9 I n general terms , the aim of the shift to weekly 
reach is to place more emphasis on attracting a wider 
range of users and as part of this the BBC is aiming to 
reach under - served audiences . But overall there appears 
to have been a lack of consistenc y in what information is 
collected and reported. T he summar y performance report 
covering the BBC as a whole provides information on 
reach among y oung people and some BBC divisions have 
data on particular social and ethnic groups . H owever , the 
BBC is currentl y in the process of defining which particular 
groups it is seeking to reach , both across the board and at 
divisional level. 
( b ) Q ualit y 
2 . 1 0 T he BBC seeks to provide high q ualit y, distinctive 
programming which it believes the public e x pects 
in return for the licence fee. Q ualit y is inherentl y 
perception- based and the BBC is seeking to supplement 
audience perceptions b y also recording its success in 
winning broadcasting awards and assessments of media 
coverage. T he wa y s the BBC currentl y measures q ualit y 
are summarised in F i g ure 6 . 
2 . 11 A lthough man y measures of q ualit y are q uantifiable, 
because of the interdependence of measures interpretation 
can be complex . F or e x ample, audience views on an 
individual programme ma y impact on wider perceptions 
of the channel concerned and of the BBC as a whole. 
A lso , audience perceptions need to be treated with some 
caution, in part because people tend not to watch or listen 
to programmes they think they will not like. T he BBC 
recognises that the results for q ualit y ma y be less clear cut 
and re q uire more interpretation than for some other areas 
where the data is harder , and it is seeking to generate 
more sophisticated data to help with assessing q ualit y 
through its new I nternet surve y ( see paragraph 3. 7 ) . 
5 The ways the BBC measures reach 
p Overall weekly reach (see paragraph 2.7) 
p Weekly reach among under-served audiences 
p The weekly reach achieved overall and by individual 
television channels (for all homes and for homes with digital 
television) and radio services 
p The average number of hours consumed per week by each 
viewer/listener/user 
p Audience share achieved overall and by individual 
television channels (for all homes and for homes with digital 
television) and radio services (see paragraph 2.6) 
Source: BBC 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
11 
( c ) I mpact 
2 . 12 T he BBC describes impact as ‘ enriching the lives 
of individuals and citi z ens’ and the wa y s it is currentl y 
measured are summarised in F i g ure 7 . 
2 . 13 T he strength of impact as a driver of performance is 
that it seeks to assess whether the BBC is delivering against 
its public purposes ( as set out in F igure 2 ) . F or e x ample, to 
help assess whether it is ‘ supporting informed citi z enship’ , 
the BBC asks audiences for their views on statements 
such as “ the BBC helps people understand and make up 
their minds ” and “ the BBC helps me understand what is 
happening in the U nited K ingdom and the world” . 
2 . 14 I mpact is , however , the driver of public value that is 
most difficult to measure and a particular challenge is the 
q uestion of attribution. F or e x ample, if the BBC concludes 
that public awareness of particular issues covered in 
BBC broadcasts has increased, it also needs evidence of 
its contribution to that change. A s with q ualit y, the BBC 
e x pects its new I nternet surve y ( see paragraph 3. 7 ) to 
generate improved data to help with assessing impact . 
( d ) V alue for money 
2 . 15 Clearl y the need to measure the value for money 
provided to licence fee pa y ers is not a new concept and 
it is a q uantifiable measure of the BBC’s performance . 
T he wa y s value for money is currentl y measured are 
summarised in F i g ure 8 . T he indicators are largely 
cost- based, focusing on inputs rather than wider concepts 
of value for money, although the BBC also measures 
audiences’ perceptions of the value for money it provides . 
6 The ways the BBC measures quality 
p Audiences’ approval ratings for the BBC as a whole, 
assessed on a scale of one to ten 
p Audiences’ perceptions of the BBC as a whole based 
on their responses to statements such as “the BBC is 
independent and impartial” and “I think the BBC makes 
great programmes to watch and listen to” 
p Audiences’ perceptions of individual television channels 
based on their responses when asked which channel is 
‘best for’ particular types of programme, such as soap 
operas or children’s programming 
p Audiences’ approval ratings for individual radio services, 
assessed on a scale of one to ten 
p Appreciation indices by television channel based on 
viewers’ responses to individual programmes, assessed on 
a scale of one to ten 
p The number and percentage of broadcasting awards won 
p The number of press articles, split between those that are 
positive about the BBC and those that are negative 
p A shortlist of programmes that BBC channel and network 
controllers consider to be especially distinctive or innovative 
Source: BBC 
7 The ways the BBC measures impact 
p Audiences’ views on the memorability of programmes, 
based on the number and proportion of respondents saying 
that a programme ‘stood out’ 
p The ‘impact factor’ – the percentage share of memorable 
programmes on a particular channel compared with its 
audience share 
p Audiences’ perceptions of the BBC and its impact on them, 
based on their responses to statements such as “my local 
interests and concerns are well reflected by the BBC’s 
coverage” and “I’ve learnt new skills or developed 
existing ones” 
p The volume and favourability of press articles 
p Assessment of the impact of the BBC’s social and 
educational campaigns 
Source: BBC 
8 The ways the BBC measures value for money 
p Cost per viewer, listener or user 
p Cost per viewer or listener hour (or cost per user reached 
in the case of online services) – calculated by dividing 
the total programming cost by the number of viewing or 
listening hours 
p Audiences’ views on: the value for money of the licence 
fee; how wisely the BBC spends the licence fee; whether 
the BBC is well run; and whether the BBC is accountable to 
licence fee payers 
Source: BBC 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
1 2 
2 . 1 6 A measure of value for money, which is not currentl y 
covered b y the performance measurement framework , 
is the proportion of its resources that the BBC spends on 
programming. A s shifting resources awa y from overheads 
into programming is a priorit y for the BBC, reflecting this 
in the measures of value for money would be a logical 
step. I t would also be consistent with the approach 
adopted b y some public service broadcasters overseas . 
F or e x ample, the A ustralian broadcaster ( A BC) reports on 
overhead e x penditure as a percentage of gross e x penditure 
as one of a number of measures of efficienc y. 
2 . 1 7 T o date performance against the value for money 
driver has been assessed less consistentl y than the other 
drivers, with cost information not presented each q uarter 
and in some divisions only at the end of the financial y ear , 
making it more difficult to track performance . 
D oes the performance measurement 
framework focus on the BBC’s 
ob j ectives ? 
2 . 1 8 T he performance measurement framework should 
be firmly linked to the BBC’s ob j ectives relating to the 
deliver y of public service broadcasting. I n ‘ Building 
Public V alue’ ( see paragraph 1.3 ) , the BBC stated that the 
Board of G overnors would draw heavily on the drivers of 
reach , q ualit y, impact and value for money to set annual 
ob j ectives for the BBC and its services , and would re q uire 
BBC Management to report against these areas . 
2 . 1 9 A t A pril 2005 the BBC had not e x plicitl y linked its 
public service broadcasting ob j ectives to the performance 
measurement framework , although at divisional level 
ob j ectives for 2005-0 6 were being framed in these terms . 
Clearl y the BBC will need to ensure that the detailed 
measures within the framework keep step with overall and 
divisional ob j ectives as they are developed. 
2 . 2 0 L inked to ob j ectives is the need to consider the 
setting of performance targets. A t present the BBC sees 
reach as the area most susceptible to target setting 
given its generally accepted goal for the organisation 
as a whole of achieving weekly reach of 95 per cent 
across the U nited K ingdom population. S ome public 
service broadcasters overseas – for e x ample the F innish 
broadcaster , Y leisradio O y – alread y have targets for reach . 
H owever , the BBC is setting annual targets across all four 
performance drivers as divisions frame their ob j ectives for 
2005-0 6 in this wa y. 
D oes the performance measurement 
framework cover all the BBC’s public 
service broadcasting? 
2 . 21 T he BBC is apply ing the performance measurement 
framework in full in si x of its eight broadcasting divisions 
which are funded b y the television licence fee to provide 
public service broadcasting4 ( F i g ure 9 ) . T hese si x divisions 
are responsible for commissioning and scheduling the 
BBC’s programming and are able to influence all of the 
performance drivers of reach , q ualit y, impact and value 
for money. 
4 T he ninth broadcasting division, BBC W orld S ervice and G lobal N ews, is not funded b y the television licence fee. T he BBC W orld S ervice is funded b y the 
F oreign and Commonwealth O ffice and is sub j ect to separate arrangements for performance measurement ; and G lobal N ews is the broadcasting arm of 
BBC W orld, which is a subscription- based service . 
9 The BBC divisions which provide public 
service broadcasting 
p Television 
p Radio and Music 
p New Media 
p News 
p Sport 
p Nations and Regions 
p Factual and Learning 
p Drama, Entertainment and CBBC 
Source: BBC 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 
part two 
1 3 
2 . 22 T he remaining two broadcasting divisions – F actual 
and L earning, and D rama , E ntertainment and CBBC 
– are b y and large non- commissioning divisions , which 
supply programming to the other divisions and have less 
influence over the performance drivers. T hese divisions 
have their own performance measurement arrangements 
focusing mainly on the q ualit y of their output, and in 
addition relevant aspects of their performance are covered 
b y the framework b y being included in the reports for the 
T elevision and N ew Media divisions . 
D o the performance measures 
allow the BBC to benchmark its 
performance with others? 
2 . 23 A useful wa y for organisations to assess their 
performance is b y comparing it with the performance 
of similar organisations , and in line with good practice 
the BBC compares aspects of its performance with that 
achieved b y other broadcasters. F or e x ample: 
p in reporting the weekly reach and audience share 
achieved b y its television channels , the BBC also 
presents data for other U nited K ingdom channels ; 
p on q ualit y, the ‘ best for’ assessments ( where 
audiences are asked which television channel is best 
for particular ty pes of programme) are b y their nature 
comparative and the BBC presents data for all the 
relevant channels ; 
p for reach and q ualit y in radio, the BBC compares the 
performance of its services with that of commercial 
radio stations . 
2 . 24 A lthough value for money also lends itself 
to performance benchmarking, the BBC and other 
broadcasters do not share data on costs due to 
competition constraints and concerns about the difficult y 
of achieving like for like comparisons . 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 4 
PART T H REE 
T he data sy stems supporting the performance 
measurement framework 
part three 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 5 
3 . 1 G ood q ualit y and reliable data is essential to an 
effective performance measurement framework . T his 
part of the report considers the BBC’s arrangements for 
generating performance data , in particular whether: 
p sy stems are in place to generate comprehensive 
and timely data ; 
p the BBC has adeq uate q ualit y assurance arrangements. 
A re sy stems in place to generate 
comprehensive and timely data ? 
3 . 2 T he BBC has sy stems for generating data for each 
of the four drivers in the performance measurement 
framework . S ome of these sy stems are longstanding 
and others are being put in place to improve the q ualit y 
and timeliness of the data . A nd increasing media 
fragmentation, with more people accessing BBC services 
b y new means , will present further challenges in terms of 
audience research in the future . 
( a ) R each 
3 . 3 A t the core of the data sy stems for measuring reach 
are two well established industry - wide sy stems which 
generate audience data – the Broadcasters’ A udience 
R esearch Board for television and R adio J oint A udience 
R esearch L imited for r a dio ( F i g ure 1 0 o v erleaf ) . T he BBC 
is a founder member and ma j or shareholder of these two 
s y stems , and has representatives on the boards of both . 
3 . 4 I n addition to collecting data on television and radio 
reach , the BBC has commissioned an I nternet research 
compan y ( N ielsen//N et R atings ) to provide information on 
the volumes of people accessing its online services . U sing 
a panel of some 5 ,000 people, the compan y measures 
usage of both the BBC’s online site ( bbc. co . uk ) and other 
websites , allowing comparisons to be made. T he data 
covers only I nternet usage at home and the BBC has also 
commissioned the British Market R esearch Bureau ( B MR B ) 
to track I nternet usage at home, work and school. B MR B 
conducts two surve y s – one of 8 ,000 people aged 1 5 and 
over ever y month and a second of 5 00 children aged 
between seven and 14 ever y q uarter . T he BBC also now 
collects data on the number of people using BBCi , 
its interactive services . 
part three 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 6 
part three 
( b ) Q ualit y and impact 
3 . 5 T he BBC uses a large number of measures to assess 
the q ualit y and impact of its public service broadcasting. 
T he key data sy stems for these performance drivers are 
two surve y s which generate data on audience perceptions . 
3 . 6 T he ‘ p a n - BBC tra cking stud y’ is conducted on behalf 
of the BBC b y the market research compan y, T a y lor N elson 
S ofres , and measures audience approval and attitudes 
towards the BBC and its services overall. I t is a continuous 
q uantitative surve y, consisting of 7 00 face - to - face 
interviews each month amongst a sample of people aged 
1 5 and over across the U nited K ingdom. A different 
sample of people is interviewed each month . Children 
aged under 1 5 are not part of the surve y and the approval 
ratings for children’s channels ( CBeebies and CBBC) are 
given b y parents. T he data is available to the BBC around 
three weeks after the end of each surve y month . 
3 . 7 T o strengthen its data collection, in A pril 2005 the 
BBC introduced a new I nternet surve y carried out b y the 
market research compan y, the G f K G roup , to generate data 
on audience perceptions of individual programmes . T he 
approach is based on e x perience in the N etherlands where 
the G f K G roup has been running an I nternet surve y for the 
public service broadcaster , Publieke O mroep, since 2002. 
3 . 8 T he surve y, which replaced a previous diar y postal 
surve y, asks respondents to rate the previous da y ’s 
programmes . I t uses a panel of 1 5 ,000 people ( up from 
the sample of 4,000 previousl y used) , of whom an average 
of 5 ,000 complete the surve y ever y da y, and there is a 
separate panel of 1,5 00 children. T he surve y provides 
more timely data with the results for each programme 
available within 3 6 hours of transmission rather than 
1 6 da y s as was previousl y the case . 
1 0 How data on television and radio reach is generated 
T elevision 
p The Broadcasters’ Audience Research Board (BARB) 
was established in 1981. It is a non-profit making limited 
company owned by the BBC, ITV, Channel 4, five, BSkyB 
and the Institute of Practitioners in Advertising. BARB 
commissions contractors to produce audience ratings 
on its behalf. 
p Reach (in-home viewing, not out-of-home viewing) is 
measured electronically using a nationally representative 
panel of 5,100 homes (representing around 11,500 
viewers). The television sets and video recorders within 
each home are electronically monitored by a meter, which 
automatically collects information about which programmes 
and channels residents and guests are viewing. The 
audience data is detailed in that it is produced on a 
minute-by-minute basis for all channels received within 
the United Kingdom, and timely in that viewing figures 
are available to the BBC the morning after programmes are 
transmitted. Consolidated data including figures for video 
cassette recorder playback is available a week later. 
R adio 
p Radio Joint Audience Research Limited (RAJAR) was 
established in 1992. It is owned by the BBC and the 
Commercial Radio Companies Association. RAJAR employs 
a contractor to produce audience ratings on its behalf and 
generates data via a nationally representative sample of 
people keeping a written diary of their radio listening 
for seven days. Approximately 130,000 diaries a year 
are completed and people are asked to record which 
stations they listened to at what times, and where they 
were listening. 
p Every three months RAJAR releases audience figures for 
each station. In January 2005 RAJAR published the results 
of an industry-wide consultation on the issues facing 
radio measurement, and one of the conclusions was that 
electronic measurement should be introduced at some 
point in the future. This could involve the use of audiometers 
(also called radiometers), which are small portable devices 
designed to ‘hear’ any broadcasts that the wearers are 
exposed to, and would result in more timely availability 
of data. 
National Audit Office , dra w ing on infor m ation fro m B A R B and R A J A R 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 7 
part three 
3 . 9 I n implementing this new approach to measuring 
audience perceptions , the BBC is giving particular 
attention to ensuring that there are enough respondents 
– it is looking to sustain the current daily sample of 
5 ,000 respondents – and that the sample is representative 
given that the I nternet is less accessible to some sections 
of the population. T he BBC is also considering broadening 
the scope of the surve y to cover da y time as well as peak 
evening programmes and more radio stations ( at present 
the surve y covers only R adio 4 ) . 
3 . 1 0 I n addition to its ongoing audience surve y s , the 
BBC commissions bespoke rese a rch to evaluate the 
impact of its periodic social and educational campaigns . 
T his ma y involve tracking, for e x ample, the number of 
people who accessed the programme’s website or j oined 
a discussion forum . 
( c ) V alue for money 
3 . 11 T he data used to assess performance against the 
cost- based measures of value for money comes from the 
BBC’s established financial reporting sy stems . E ach BBC 
division prepares monthly reports and accounts, including 
this cost data , which are anal y sed b y the central F inance 
division. D ata for the audience perception measures of the 
value for money of the BBC is generated b y the pan - BBC 
tracking stud y ( see paragraph 3. 6 ) . 
D oes the BBC have adeq uate q ualit y 
assurance arrangements? 
3 . 12 Much of the data the BBC uses is generated b y 
established e x ternal sources ( including B A R B , R A J A R 
and market research companies ) and it is then sub j ect 
to processing within the BBC. U sers of the performance 
measurement framework need to be confident that the 
source data is reliable and that it is reported accurately, 
and responsibilit y for providing this assurance rests with 
the BBC. 
3 . 13 T he BBC relies on e x ternal data providers’ sy stems 
of q ualit y assurance and its I nternal A udit team carries out 
spot checks on the non- financial figures reported in the 
BBC’s annual report. A t present , however , the BBC does 
not have q ualit y assurance arrangements which focus 
e x plicitl y on the performance measurement framework 
to provide assurance that source data is reliable, used 
appropriately, reported accurately and interpreted fairl y. 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 8 
PART F OUR 
H ow performance information is being used 
and reported 
part four 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 1 9 
4 . 1 O nce performance information has been gathered 
and validated, the overall success of a performance 
measurement framework lies in how it is used. T his part of 
the report looks at how the BBC is using and reporting the 
information generated b y its performance measurement 
framework . I n particular we considered whether: 
p the performance measurement framework is being 
used to influence thinking in the BBC; 
p the reporting arrangements are effective ; 
p the performance measurement framework provides a 
basis for the BBC to be e x ternally accountable. 
I s the performance measurement 
framework being used to influence 
thinking in the BBC? 
4 . 2 I t is earl y da y s in the life of the new performance 
measurement framework but from our discussions with 
BBC S enior Managers and other staff it was apparent that 
reach , q ualit y, impact and value for money – the drivers of 
the framework – are becoming common currenc y in the 
BBC’s thinking about public service broadcasting. 
A nd as highlighted in part 2 of this report, the performance 
measurement framework is being incorporated into the 
BBC’s internal ob j ective setting process and in this wa y 
reach , q ualit y, impact and value for money are being 
cascaded throughout the organisation. 
4 . 3 E mbedding reach , q ualit y, impact and value for 
money throughout the organisation is however , as the BBC 
recognises , a longer term challenge. T o help with this , 
the reports reflecting the information gathered under the 
performance measurement framework are made available 
to staff via the BBC intranet . T he BBC’s aim of embedding 
the framework across the organisation underlines the 
importance of clear definitions to help achieve consistent 
use and understanding ( see paragraph 2.4) . 
4 . 4 T he BBC divisions we spoke to found the reports 
produced under the performance measurement 
framework a useful tool to inform their ongoing business 
management . A nd although it is earl y da y s , there was 
evidence of the BBC using the framework to influence 
decisions . F or e x ample: 
p the E x ecutive Board is using the performance 
measurement framework to inform future funding 
allocations for the broadcasting divisions , with 
programming plans being prepared and then 
assessed in terms of their contribution to reach , 
q ualit y, impact and value for money ; 
p the T elevision division has used the performance 
measurement framework to review its programming 
and inform decisions on the allocation of funding. 
R esource bids have been presented in reach , q ualit y, 
impact and value for money terms , making it easier 
to compare them; 
p the N ews division has used the performance 
measurement framework to help decide whether 
particular programmes should be rescheduled, 
repeated or re - commissioned. I t has found the 
ob j ectivit y and evidence - based approach of the 
framework to be a useful support tool in making 
such decisions . 
part four 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 20 
part four 
A re the arrangements for reporting 
performance effective ? 
4 . 5 Performance reports, reflecting the information 
gathered under the performance measurement framework , 
are produced q uarterl y for the BBC Board of G overnors, 
the E x ecutive Board , the Creative Board and the 
J ournalism Board , which oversee the BBC’s public service 
broadcasting ( see F igure 1 ) . A ll these Boards receive 
a summar y report covering the BBC as a whole and a 
detailed report on the performance of individual divisions , 
including a divisional summar y. I n line with good 
practice , the reports present performance over time, with 
data for previous q uarters and performance compared 
y ear - on- y ear . 
4 . 6 T he BBC previousl y reported performance ever y 
month but, in the light of concern that this fre q uenc y 
risked diminishing the impact of the reports and 
producing ‘ knee- j erk’ reactions to short term fluctuations 
in performance , it has now moved to q uarterl y reporting. 
T he staff we spoke to felt this change had led to more 
meaningful discussion, and there was still the 
opportunit y to raise pressing performance issues at 
monthly Board meetings . 
4 . 7 T he BBC is seeking to improve performance 
reporting in the light of feedback from users and we 
identified a number of wa y s in which the reports ( in 
particular the summar y report) could be sharpened and 
presentationally made more user - friendly. F or e x ample: 
p there is a need to differentiate between priorit y 
information and material that is of secondar y 
importance . A t present the performance reports 
include a large number of measures , which are 
not prioritised for the reader ( for e x ample, the 
F ebruar y 2005 summar y report included over 8 0 
individual measures presented graphically plus 
other q ualitative information) . I t would be useful 
for the reports to highlight a smaller number of key 
measures across the four drivers ( which ma y var y 
for different audiences ) , with the less important 
information presented separately ; 
p there is scope for providing users with clear 
definitions of the performance measures and the 
methods b y which they are calculated; 
p the reports could highlight more clearl y key issues 
for discussion b y supplementing the graphical 
presentation of data with more e x planation. F or 
e x ample, the N ews division has introduced a ‘ talking 
points’ section at the front of its performance reports, 
flagging up areas for discussion; 
p the reports could set out more clearl y e x pected 
levels of performance to help readers interpret the 
information. A lthough the reports use a ‘ traffic light’ 
s y stem to signal whether performance is average, or 
above or below average, the parameters for these 
j udgements are not made clear and therefore there is 
a risk of inconsistenc y. 
4 . 8 A t present the performance reports are not tailored 
to different audiences – the Board of G overnors and the 
S enior Management Boards receive the same reports. 
I n the case of the G overnors, the G overnance U nit is 
considering how the summar y report could be better 
tailored to the specific needs of the G overnors and 
supplemented b y briefing papers to highlight key issues . 
T he G overnance U nit is also reviewing its arrangements 
for sy stematic follow - up of issues raised b y the G overnors 
to ensure the arrangements are as effective as possible. 
D oes the performance measurement 
framework provide a basis for the 
BBC to be e x ternally accountable? 
4 . 9 ‘ Building Public V alue’ ( see paragraph 1.3 ) envisaged 
the performance measurement framework enabling the 
Board of G overnors, BBC Management and the public to 
understand better how well the BBC’s services are doing. 
T he document stated the G overnors’ intention to use the 
measures of reach , q ualit y, impact and value for money 
to set ob j ectives for the BBC and its services each y ear , 
and that they would form the basis for the BBC’s A nnual 
R eport to Parliament and the British public . 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 2 1 
4 . 1 0 I n taking forward the e x ternal reporting dimension 
of the performance measurement framework , a key 
q uestion for the BBC will be how to present the 
information in an understandable and manageable 
form given the volume of data involved. T here are also 
technical challenges since e x ternal reporting ma y involve 
the BBC aggregating and interpreting the results of a wide 
range of performance measures . A t the same time the BBC 
will need to ensure that the integrit y of the measurement 
framework is maintained. 
4 . 11 T he F innish broadcaster ( Y leisradio O y ) publishes 
an annual audience report, alongside its annual report, 
which sets out the e x tent to which it has succeeded 
in providing universal television, radio and online 
services , and “ discusses F innish people’s perceptions and 
e x pectations of public service broadcasting as well as 
their satisfaction with the service ” . T he report provides 
both narrative and statistical information on areas such 
as reach and share , satisfaction with programming, and 
perceptions of value for money. T he N etherlands public 
service broadcaster ( Publieke O mroep) plans to use its 
performance q ualit y card ( see paragraph 1.10 ) as an 
instrument for e x ternal accountabilit y. 
part four 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 22 
1 O ur stud y focused on the new performance 
measurement framework which the BBC began to 
introduce in A utumn 2004. T he BBC is continuing to 
develop and refine the framework so our aim was to 
review progress and contribute to the BBC’s thinking on 
the further work to be done. 
2 I n doing our work , we were guided b y the good 
practice principles set in ‘ Choosing the R ight F A B RIC – a 
framework for performance information’ , produced j ointl y 
b y H MTreasury, the Cabinet O ffice , the N ational A udit 
O ffice , the A udit Commission and the O ffice for N ational 
S tatistics . T he publication identifies the key components 
of a good performance measurement sy stem. 
APPEND IX 
S tud y methods 
appendix 
C omponents of 
performance measurement 
M onitoring 
Comparing performance 
over time or with other 
organisations 
E valuation 
of the performance 
information system itself 
S trate g y 
Aims, objectives and 
action plans 
M easures 
Aligned with 
strategic objectives 
T ar g ets 
Setting intended levels 
of performance 
V erification 
Through internal or 
external means 
R esults 
Actual performance 
achieved 

PUBLIC SERVICE BROADCASTING: THE BBC’S PERF ORMANCE MEASUREMENT F RAMEWORK 2 3 
W ork at the BBC 
I nterviews with BBC staff 
3 W e met the D irector - G eneral and D eputy D irector - 
G eneral of the BBC to get their views on the performance 
measurement framework . 
4 W e carried out interviews with key staff from the 
BBC divisions involved in implementing and supporting 
the performance measurement framework , including the 
S trategy and D istribution division and the Marketing, 
Communications and A udiences division. T he interviews 
covered the design of the framework , the sy stems for data 
collection, and the performance reporting arrangements. 
5 W e interviewed staff from three of the BBC’s 
broadcasting divisions – T elevision, N ews and S port 
– which we used as case studies of how the performance 
measurement framework is being implemented in practice 
and how it is being used to influence planning and 
management . W e also sought their views on how the 
framework might be improved from the perspective of the 
broadcasting divisions . 
6 W e interviewed key staff from the BBC’s G overnance 
U nit , which supports the Board of G overnors in its work , 
to get their views on how the performance measurement 
framework is being used and how it might be improved 
from the perspective of the G overnors. 
R eview of performance reports and 
other material 
7 W e reviewed the q uarterl y reports that are produced 
under the performance measurement framework – both 
the summar y reports covering the performance of the BBC 
as a whole and the more detailed divisional reports which 
feed into and supplement the summar y reports. 
8 W e also e x amined a range of other published and 
unpublished material , including ‘ Building Public V alue’ , 
the BBC’s A nnual R eport, minutes of meetings of the BBC’s 
Board of G overnors and Management Boards , divisional 
ob j ectives and audience research material . 
I nterviews with stakeholders 
9 W e met officials from the D epartment for Culture , 
Media and S port with responsibilit y for broadcasting 
polic y to discuss the BBC’s approach to performance 
measurement , the G overnment’s G reen Paper on the 
R eview of the BBC’s R o y al Charter , and wider matters 
relating to public service broadcasting. 
1 0 W e interviewed staff from O fcom ( the independent 
regulator of the U nited K ingdom communications 
industries ) to discuss O fcom’s review of public 
service television broadcasting, the BBC’s approach to 
performance measurement , and the alignment between 
the two . 
11 W e met the team from S pectrum S trategy 
Consultants, commissioned b y the Board of G overnors 
to assess the BBC’s proposed ‘ public value test’ ( see 
F igure 3 ) , to discuss our and their work . 
12 W e met a number of other parties with e x perience or 
e x pertise in public service broadcasting to get their views 
on the BBC’s approach to performance measurement and 
wider matters. W e interviewed: Patrick Barwise , Professor 
of Management and Marketing at the L ondon Business 
S chool; Philip Booth , E ditorial D irector at the I nstitute of 
E conomic A ffairs; J amie Cowling, then Media R esearch 
F ellow at the I nstitute of Public Polic y R esearch ; and 
Claire G rimmond, H ead of R esearch at Channel 4. 
R esearch on approaches to 
measuring public service 
broadcasting overseas 
13 W e looked at how public service broadcasters in 
other countries are approaching performance measurement . 
A lthough comparing public service broadcasting in 
different countries is problematic because structures , 
funding sy stems and remits var y from one country to the 
nex t , we wanted to see in general terms how the BBC’s 
performance measurement framework compared with 
arrangements being implemented overseas . 
14 W e carried out research and liaised with 
representatives from the E uropean Broadcasting U nion and 
with parties with knowledge or e x perience of public service 
broadcasting in other countries , including broadcasters, 
industry organisations and academics . W e also met the 
BBC’s H ead of E uropean Polic y and representatives from 
the N ordic public service broadcasters. 
15 T hrough these contacts, we gathered information 
on the approaches to performance measurement in the 
following countries: A ustralia , Belgium , D enmark , F inland, 
G erman y, the N etherlands , N orwa y and S weden. 
appendix 



British Broadcasting Corporation 
Broadcasting House 
London W1A 1AA 
This report is available online at 
bbcgovernors.co.uk