Best phone network 2017 UK: Which is the best mobile network, best 3G, best 4G in the UK
With the Pebble Network now entering the mobile market, there’s a lot of choice for UK consumers when it comes to choosing a phone network. We take a look at the competition, independent tests and more to decide which is the best network in the UK.
What's the UK's best phone network in 2017? Read on to find out the pros and cons as we compare the best mobile networks
Choosing a smartphone or tablet is just the start. In this world of constant connectivity the quality and speed of your chosen network is critical to your ability to work and play on the hoof. But which mobile network is best? As friends and colleagues and you will hear multiple answers. Connectivity can vary from locale to locale, and day to day. Visit the websites of all the major players and they will tell you that they have great coverage, everywhere. The truth is more nuanced. Also see: Best MiFi 2017.
Interestingly, according to StatCounter for the first time in October 2016 more people accessed the internet on a mobile phone or tablet than they did on a desktop or laptop PC. It recorded 51.2 percent mobile against 48.7 percent desktop, making it all the more important that you choose the best mobile network.
Latest: O2, EE & Vodafone to increase prices
Some of the UK's networks are increasing their rates as per the Retail Prices Index (RPI) rate of inflation, which means customers may see a bigger bill when it arrives in April. O2 says that it is applying a 2.6% increase, EE is applying a 2.5% increase and Vodafone is yet to decide the percentage but has confirmed that it will be doing so for customers who upgraded or joined after 6 May 2016.
Here we take a quick look at each operator and what they offer right now. They all offer 4G data services.
Best phone network 2017 UK: BT Mobile
BT is fairly new to the market, and uses its branded services on EE’s UK network. It’s a good option if you already have BT Broadband as you can apply for a £5 monthly discount on a contract with a handset.
It also offers SIM-only plans for pretty decent prices. As it’s on EE’s network (which itself is a combination of the old UK networks T-Mobile and Orange) you’re likely to get excellent coverage and good download speeds.
Best phone network 2017 UK: Pebble
Pebble is becoming a bit of a surprise in the UK mobile market. It sprung up as if from nowhere just before last year was the first time we saw it, but that won’t really affect your choice of network unless you need the ultimate mobile coverage nearly everywhere. Pebble uses all the networks coverage nationally, with very few blackspots as a result.
Its phone contract prices can be a tad pricey, but for sheer confidence in coverage it’s a solid choice.
Owned by O2, Giffgaff is a little operator but one with attractive SIM-only deals. Using its owners network, it offers 2GB of data on plans from as little as £10 a month. It’s a fine choice if you already own a handset – just get a SIM and you’re good to go. However it does offer phones on a contract if you need both.
Giffgaff won consumer watchdog uSwitch’s Network of the Year 2016, so it’s clearly doing something right.
O2 is a brand that has been around since 2002 in the UK and has been owned by Telefonica since 2005. Recently rival operator Three attempted to merge with O2 but the European Commission opted to block the move (this was in the wake of the BT and EE sale).
Despite this, O2 is one of the most well-known operators around. Its prices are high compared to some competitors but coverage is excellent and its pay as you go range is one of the best in the country.
Best phone network 2017 UK: Sky Mobile
The newest addition to the UK mobile market, Sky has become a ‘quad-play’ provider by adding mobile to its TV, broadband and landline offerings.
Existing Sky customers get the best deals, with free calls and texts. You just pay for how much data you want, on a plan that uses O2’s network. Non-Sky customers can pay £10 extra per month for unlimited calls and texts. If you want to use Sky for everything, now you can.
Best phone network 2017 UK: Tesco Mobile
Another user of O2’s big UK network, Tesco Mobile provides contracts, pay as you go and SIM-only deals at decent prices. Again, because it uses O2’s network coverage will be decent and often the subsidies for new handsets are good.
SIM-only prices can be under £10 per month, so if you’ve got a handset you already love it could save you a lot of money.
Best phone network 2017 UK: Three
Three is unique in that it offers reasonably priced ‘all you can eat’ data bundles. That’s right, unlimited data. If you can afford it, you can get the best handsets in the land with unlimited Internet access.
Pay as you go rates are also pretty decent if you don’t need as much data, minutes or texts. Three unsuccessfully attempted to buy O2, and the fact it didn’t is good for the mobile market. More competition means better prices for you the consumer.
Best phone network 2017 UK: Virgin Mobile
Virgin was late to the party with 4G, but has caught up and is an attractive option if you already get your broadband connection and TV from Virgin, cutting down the number of tech providers you gave to deal with.
Best phone network 2017 UK: Vodafone
Vodafone is a staple brand in the UK mobile market, and its coverage is excellent. We will say that in our own personal experience and also in the media recently it has suffered thanks to bad customer service. However, the actual network itself provides incredibly reliable fast 4G data connections and the SIM-only deals are well priced.
You may want to shop around if you feel that the contract prices are too high; they often are around the launch of a handset but if you can wait a few weeks they generally come down.
You'll also like: Best SIM-only deals.
Best mobile network UK Awards 2016
In February, uSwitch unveiled the winners of its Mobile Awards 2016, voted for by more than 5,000 consumers and a judging panel of journalists.
Giffgaff was declared the best mobile network, with Tesco Mobile in second place and Three in third. Vodafone was the only one of the big UK mobile networks not to scoop a first-place award.
Read on to discover how Pebble, EE, Three, O2, Vodafone, Giffgaff, Tesco Mobile and Virgin fared when we took a look.
Best Mobile Network of the Year 2017
2. Tesco Mobile
Best Pay Monthly Network 2017
2. Tesco Mobile
Best PAYG Network 2017
Best SIM-only Network 2017
2. Tesco Mobile
Best for Roaming 2017
Best Customer Experience 2017
Best for Data 2017
Best Network Coverage 2017
1. Pebble Network
Network Customer Service 2017
Best Value Pay Monthly 2017
3. Virgin Media
Best Value SIM Only 2017
2. Tesco Mobile
3. Virgin Media (Also see our Best SIM-only deals 2017 article.)
Fastest Mobile Network powered by RootMetrics 2016
Best mobile network UK: Ofcom research 2016
On 31 March, Ofcom published its research into the performance of the UK's big four networks: EE, O2, Three and Vodafone. The research was carried out between November and December of 2015 in five UK cities: Cardiff, Edinburgh, Liverpool, London and Norwich.
The researched focused on two key metrics: download speed and web browsing performance.
Overall, EE came out on top when it comes to both web browsing speed and download speed, with an average page load speed of five seconds, a 98% successful loading rate, 20Mbit/s average download speed and 92% speeds over 2Mbit/s.
If you live in Norwich, however, Three managed to take the top spot as the fastest network there.
Below is the full breakdown of the overall results, and you can find out more in Ofcom's report here.
In addition to Ofcom's performance research, the organisation has also shared data on the volume of customer complaints it received against major mobile network providers.
Tesco Mobile has the least complaints per 100,000 customers, but it is likely to have the least customers too so it's tricky to make any solid conclusions from the report.
We can see, however, that complaints against Vodafone have shot up drastically since the middle of 2015, after customers reported incorrect billing.
Having said that, in April 2016 Vodafone introduced a unique 30 day network guarantee. This allows all new and upgrading customers 30 days to cancel their contract from the start date. This is good if you want to try how Vodafone is for coverage in your area, or if the deal you have picked is right for you.
This also applies to small businesses buying from Vodafone, and to all its bundles and devices. You can look for more information here.
Smartphone Cities provides information on mobile network performance across five UK cities: Cardiff, Edinburgh, Liverpool, London and Norwich, chosen because all four mobile network operators (EE, O2, Three and Vodafone) had rolled out 4G networks there at the time of our testing. It is important to note that the results presented in this report do not reflect UK-wide performance and are limited to the cities in which we tested. Our results provide a snapshot of performance in November and December 2015. All operators carried out varying degrees of maintenance on their networks during our test period which may have impacted on their results.1 All results were gathered on a 4G tariff, unless stated otherwise, with our five cities each having comparable levels of 4G coverage during testing. Key metrics Our report focuses on the two key metrics relevant to the consumer experience of mobile broadband - download speed and web browsing performance - before examining performance relating to other internet activities, and voice calls. For our key metrics, we present the average results across all cities, as well as the distributions, to give a more rounded view of performance. For web browsing speed, we analyse the percentage of occasions when the BBC homepage successfully downloaded within 15 seconds, as well as the average time it took for all information on the homepage to reach the user’s handset. Consumers are unlikely to notice the difference between average speeds across operators, but are likely to feel frustrated when a web page fails to download at all. For download speed, we consider the proportion of tests that produced a speed greater than 2Mbit/s in addition to the overall averages, as such speeds are likely to be sufficient to support high-capacity video services. Looking at the distribution gives a better idea of consistency of performance and how the networks respond under stressed conditions. Figure 1 outlines the headline results for our key metrics at an aggregate level across all cities, while the rest of the executive summary presents the results within each city, where there is variation in performance between the operators. Figure 1: Key metrics scorecard Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. 1 Further detail on this is provided in the Q&A on page 12. Web browsing speed Download speed Average speed Successful loading Average speed Speeds over 2Mbit/s EE 5 seconds 98% 20Mbit/s 92% O2 6 seconds 89% 10Mbit/s 69% Three 6 seconds 95% 15Mbit/s 87% Vodafone 6 seconds 91% 12Mbit/s 82% 3 4G v 3G 4G services provided faster download speeds than 3G services consistently and across all networks. Across all five cities and all operators, the average download speed when using a 4G network was 17Mbit/s compared to 6Mbit/s when using 3G technology. City-by-city results • EE had the fastest average download speed in Cardiff at 23Mbit/s, followed by Three (14Mbit/s), Vodafone (9Mbit/s) and O2 (6Mbit/s). • EE also had the most consistent download speed performance in Cardiff, with 95% of tests coming in at greater than 2Mbit/s, which compared to 92% on Three, 74% on Vodafone and 54% on O2. • In the web browsing test in Cardiff, the BBC homepage loaded in full on 99% of tests on EE, 97% of tests on Three, 89% of tests on Vodafone and 88% of tests on O2. • The fastest average download speed in Edinburgh came from EE at 21Mbit/s; there was no significant difference between the 15Mbit/s averages of the other three operators. • O2 (96% of speeds greater than 2Mbit/s) and Vodafone (94%) did not differ significantly when looking at the consistency of download speeds in Edinburgh, while the 92% on EE 4 was a lesser proportion than O2 but not significantly different to that of Vodafone. Three had 87% of speeds greater than 2Mbit/s in Edinburgh. • For web browsing performance, the BBC homepage downloaded in full on 96% of occasions for EE and Three, and on 92% of occasions on Vodafone and O2. • In Liverpool, EE had the fastest average download speed on a 4G tariff across the operators (20Mbit/s), while there was no significant difference between the averages of Three (15Mbit/s) and Vodafone (14Mbit/s). O2 had an average download speed of 8Mbit/s in Liverpool. • Vodafone and EE had the most consistent download speed performance in Liverpool, with 93% and 91% of download speed tests coming in at above 2Mbit/s respectively. There was no significant difference between the 89% of speeds on Three, compared to EE, while O2 had 73% of speeds above 2Mbit/s. • The BBC homepage loaded in full on 99% of tests on EE, 97% of tests on Three and Vodafone and 92% of tests on O2. • London had the slowest average download speed across all operators, across our five cities (12Mbit/s). The fastest average speed was on EE (17Mbit/s), followed by Vodafone (13Mbit/s), Three (9Mbit/s) and O2 (7Mbit/s). 5 • EE had the highest proportion of download speeds over 2Mbit/s in London, at 92%. There was no significant difference between the 81% on Three and the 80% on Vodafone, while O2 had 59% of speeds greater than 2Mbit/s in London. • EE and Three had BBC web browsing success rates of 99% and 96% respectively in London, which compared to 87% on both O2 and Vodafone. • Three had the fastest average download speed in Norwich at 22Mbit/s, followed by EE (19Mbit/s), O2 (13Mbit/s) and Vodafone (10Mbit/s). • Looking at consistency in Norwich, EE and Three had the largest proportion of download speed tests coming in at greater than 2Mbit/s; at 89% and 88% respectively. There was also no significant difference between the 70% of Vodafone and the 67% of O2. • For web browsing performance, the BBC homepage loaded in full for 96% of tests on EE, 90% of tests on Three and Vodafone and 85% of tests on O2. Other internet activities The results for a whole range of other tests relevant to the typical consumer experience of mobile network performance are found in the report and supporting annex. • EE had the fastest upload speed across all cities, at 20Mbit/s, followed by Three (12Mbit/s), and Vodafone and O2 (both 10Mbit/s). • We loaded the trailer for the latest Bond film, Spectre, to test YouTube streaming performance. On 97% of occasions the trailer loaded in HD on EE, compared with 87% on Three, 86% on Vodafone and 85% on O2, across all of our five cities. • Three had the quickest response time across all cities, important for activities that require minimal delay such as Skype and FaceTime, at 44ms, followed by EE (56ms), Vodafone (81ms) and O2 (92ms). Voice call performance Successfully setting up and maintaining a voice call remains important for many consumers using their smartphones. The vast majority of calls were successful throughout our test period, with 99% of calls being completed on EE and 98% on O2, Three and Vodafone. 6 4G in context What is 4G? 4G stands for fourth-generation mobile technology; it is the fourth version of smartphone technology. 2G technology was suitable for making calls and sending text messages, while 3G made it possible to access the internet more effectively through a smartphone. 4G services should make it much quicker to surf the web and provide the kind of broadband speeds required for video streaming, using mapping services and social networking. Smartphone society The UK is increasingly becoming a smartphone society, with 70% of all adults owning a smartphone, making the performance of mobile networks more important than ever. 2 Mobile network performance will remain vital in a market in which half of all smartphone users say they are ‘hooked’ on their mobile phone, and over half of 4G users claim to use their smartphone more than ten times a day. 3 More consumers now say that their smartphone is their most important device for connecting to the internet. The take-up of 4G services has been increasing rapidly, with the number of 4G subscribers reaching 24 million when we carried out our tests. Consumers with 4G are more likely to get the most out of their smartphones. On average, a 4G smartphone user will have 20 apps on their device, compared with an average of 14 for non-4G smartphone users. 4G users are more likely to send/receive emails, access social networks, and send photos/videos by text, use online maps, as well as bank and shop online. 4 Mobile network operators There are four mobile network operators (MNOs) in the UK which offer 4G services: EE, O2, Vodafone and Three. There are also many mobile virtual network operators (MVNOs), which are third-party companies that lease telephone and data capacity from the network operators. The results contained in this report will not necessarily correspond with the network performance of the MVNOs (e.g. our results on O2 will not necessarily be representative of performance on giffgaff) while other aspects of the mobile quality of experience, such as customer service and tariffing, will also vary between MNOs and MVNOs. The table below provides examples of which virtual operators correspond with which of the operators: 2 Ofcom consumer research, H2 2015. 3 Communications Market Report 2015, Ofcom: 246.. http://stakeholders.ofcom.org.uk/binaries/research/cmr/cmr15/CMR_UK_2015.pdf 4 Communications Market Report 2015, Ofcom: 79-83. http://stakeholders.ofcom.org.uk/binaries/research/cmr/cmr15/CMR_UK_2015.pdf 7 Figure 2: MVNO network use What about coverage? Before consumers can benefit from the performance of a 4G network, they need to be using their smartphone in an area with 4G coverage. The results in this report were gathered in cities where each operator had 4G coverage. Ofcom’sown online coverage checker maps are regularly updated with the latest coverage information supplied by the operators themselves. 5 3 4 5 6 7 8 9 5 Links to the operator’s maps, which may display different levels of coverage, can be found here: EE, O2, Three, Vodafone. Mobile Network Operators Examples of MVNOs EE BT Mobile, Virgin Mobile O2 Tesco Mobile, Lyca Mobile Three Shebang, iD Vodafone TalkTalk, Lebara Mobile 8 10 The metrics we have used Our main results were gathered on a 4G tariff for each of EE, O2, Three and Vodafone, meaning 4G was the best technology available on each smartphone. We also gathered results solely on 3G, to show the performance improvements that might typically be experienced by consumers if they upgrade from 3G to a 4G service. The two key metrics we have used are download speeds and web browsing time. Download speeds Download speed refers to the speed at which information is transferred from the internet to a device. In effect, it determines to how quickly a file can be downloaded from a remote source to a smartphone. Typically, this involves downloading a file from a website. Popular types of file that consumers download on their smartphones include music, films, pictures, applications and e-books. The unit of measurement for download speeds is megabits per second (Mbit/s). Why do download speeds matter? Download speeds matter because they determine how long consumers have to wait before getting the content on their device. Higher download speeds are particularly important for downloading larger files, such as films or apps. For the best consumer experience, getting fast download speeds consistently matters as much as the speeds themselves. The distribution of download speeds gives an indication of this, as does the time-of-day analysis presented later in the report. Examining the variation in average speeds across different day-parts gives an idea of how networks behave at busier times. Web browsing time Web browsing is an activity that allows consumers to access content through an internet browser. Many activities – from using a search engine to loading a news website – require web browsing functionality. To measure web browsing performance we look at both BBC homepage loading time and the distributions, particularly the proportion of occasions on which the web page failed to download within 15 seconds. BBC homepage load time refers to how long (in seconds) it took for the BBC homepage to load on each network, from the time between a consumer sending a request for a web page and the page finishing loading. While our results show the time it took for all of the web page to be downloaded, consumers will be likely to interact with the web page before the final piece of information reaches the handset. The BBC homepage is dynamic, meaning that the content of the page changes over time, whenever the page is updated. This means that different content was viewed over the period of the testing. We also tested a standard web page as an internet activity metric; this is explained below. 9 Why do web browsing speeds differ? Each of the networks might deal with a request to load a web page differently, and delays can be introduced at intermediate steps along the way. Therefore, web browsing will be quicker on some networks than others and quicker on some web pages than on others. The time it takes is related to download speed as well as other factors, such as how quickly the network can identify the location of the website. Network operators can use a number of different techniques to improve network performance; for example, by storing the most frequently visited websites at a location nearer the consumer, or by using compression techniques to reduce the size of an image file contained within a web page. Internet activity metrics This report also presents the results of different networks’ performance for: • upload speed; • YouTube streaming; • response times; and • standard web page loading. Voice call success rate Call success rate is the proportion of calls to a fixed landline that were successfully completed, compared with those that dropped within 90 seconds or failed to be set up at all. Additional metrics are reported in the supporting annex to this report and provide further detail on mobile network performance across our testing period. 10 Understanding the results Measuring performance There are many ways of measuring the performance of mobile networks. The results of our testing focus on download speed and web browsing time, because downloading and browsing the web are key mobile internet activities for consumers. Uploading, video streaming, and making and receiving calls are also typical elements of smartphone use on a daily basis. Most of our tests were on the performance of mobile networks on a 4G tariff. On a 4G tariff, a network will provide a 4G service if it can, and if it cannot provide a 4G service at a given time, it will provide the best alternative technology instead. Testing networks on a 4G tariff therefore means testing the performance of the operators on the best network service they can offer in at a given time at a given place. This report does not provide an exhaustive analysis of mobile networks’ performance. It gives a snapshot of how EE, O2, Three and Vodafone performed in a series of tests we conducted in November and December 2015, in certain locations, with reference to a set of metrics that are easily recognisable to consumers and relevant to how they use their smartphones. However, it should also be noted that network performance is only one of the factors that consumers should take into account when choosing their provider. Price, quality of customer service, coverage, contract terms and handset choice are also relevant and will play a part in consumers’ quality of experience. Speed and consistency Both the speed of mobile broadband services and their consistency are important as measures of performance. While consumers can feel frustration with slow download speeds and long waits for web pages to load, connection failures and failed calls are also a source of irritation. An average speed metric will indicate how fast a service is on average, but it does not reveal how frequently that service fails. In order to gauge the consistency of a mobile network’s performance, it is necessary to look at the distribution of results. Distribution charts are featured in this report so that the consistency of each operator is evident. The results: Q&A Q: Are there any measures of mobile network performance which are not included in the report? A: Not every possible measure of performance is included in this report. Metrics relating to long-form video (as might be viewed on Netflix and BBC iPlayer), and specifically to audiostreaming, are not included. 11 Q: Do the results reflect what performance is like across the UK? A: No. You should not interpret the results in this report as reflecting the performance of mobile networks in all parts of the UK. The report gives a snapshot of network performance in November and December 2015 across the five cities in which we tested. We selected Cardiff, Edinburgh, Liverpool, London and Norwich because they all had 4G network coverage from the operators. The availability of 4G services is generally more common in cities and urban areas. Our results are presented at a city-by-city level, so they will be of most use to consumers who use their smartphones principally in one or more of our five test cities. There are other sources that provide information relating to the mobile quality of service across the UK, such as RootMetrics, P3 and Ookla. Q: What can affect how well each of the networks performed in the tests? A: A range of factors can affect the performance of a mobile network in any given test. The performance of mobile networks can vary according to the load on the network at different times; i.e. how many people are using the network at once. The type of device a consumer uses also plays a role in the performance of mobile networks. We conducted the November-December testing using a Samsung Galaxy S5 (model SM-G900F), in order to reflect the type of 4G-enabled devices that were popular among consumers at the time of testing. However, consumers with the most up-to-date phones are likely to experience improved performance, as they can benefit from the latest network developments, such as the deployment of carrier aggregation technology from EE and Vodafone, and Three and EE’s introduction of VoLTE services in parts of the UK. Conversely, consumers with older phones might experience poorer performance. The operators also perform maintenance on their networks as they seek to upgrade the performance available across the UK. Over the course of our test period, there was scheduled maintenance for Vodafone in Cardiff, Norwich and Liverpool, while Three’s results in London and Edinburgh may have been affected by operational issues they experienced while we tested there. EE reported network issues in London, and O2 did most of their maintenance in Edinburgh and London during our test period. It is worth noting that maintenance is an ongoing process for all operators, and would have an impact on whichever test period we had chosen. Q: Was it only 4G network performance that was tested? A: No. Most of this report relates to the performance of mobile networks where 4G is the best available technology. In practice, this means that most of our testing was on 4G when possible, dropping down to a lower technology when 4G was not available. In addition, we did a small number of tests where we compared the performance of 4G to 3G, in order to highlight the performance benefits available to consumers who may wish to upgrade. These tests are presented in the key metrics section of our report. Q: What about coverage? 12 A: Not all consumers in the UK will be able to access the 3G and 4G services of EE, O2, Three and Vodafone, because the areas where they live might not be covered by these networks. We chose our test cities because each operator had 4G coverage in all of them. Ofcom’s interactive coverage maps provide the latest coverage figures for each operator. Q: What does ‘statistical significance’ mean? A: If a result is statistically significant, it means that the result would be highly likely to be the same even if the test were to be repeated on a much larger scale. Significant difference tables are used throughout this report. These tables indicate when results are statistically significant against one another. Significant difference tables are essential in comparing the performance of operators against each other. To reflect the experiences of consumers more generally, results must be statistically significant. However, not all statistically significant differences are noticeable to the consumer. For instance, a difference of 0.2 seconds between two networks, for the time it takes to load a web page, might be statistically significant, but is unlikely to be noticeable to consumers. 13 Results: download and web browsing performance Download results Overall, EE achieved the fastest average download speeds, at 20Mbit/s on average across our five cities. EE also achieved the fastest average download speeds in four of the five cities individually. In Norwich, Three provided the fastest average download speeds, at 22Mbit/s. O2 had the slowest average speeds across the five cities, and in Cardiff, Liverpool and London individually. Figure 3: Average 4G tariff download speed, by city Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. Figure 4: Average 4G tariff download speed: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. 14 13 16 14 12 16 20 23 21 20 17 19 10 6 15 8 7 13 12 9 15 14 13 10 15 14 15 15 9 22 0 5 10 15 20 25 Overall Cardiff Edinburgh Liverpool London Norwich All EE O2 Vodafone Three Mbit/s Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall EE Three Vodafone O2 Cardiff EE Three Vodafone O2 Edinburgh EE O2, Three, Vodafone (=) - - Liverpool EE Three, Vodafone (=) - O2 London EE Vodafone Three O2 Norwich Three EE O2 Vodafone These tables show how operators performed against each other 14 Across all tests, 3% of file downloads failed to start across all operators. The proportion of downloads with speeds of >2Mbit/s varied significantly across operators, accounting for 92% of downloads attempted on EE’s network – the highest proportion of all operators. In contrast, O2 had the lowest proportion of test samples with speeds of over 2Mbit/s, at 69% across all cities. Figure 5: Distribution of 4G tariff download results Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. The variations in distributions across each city are summarised in the executive summary; more in-depth figures are provided in the chart pack accompanying this publication. 3G versus 4G: download speed results 4G services provided faster download speeds than 3G services, consistently, across all networks. There was no significant difference between the average 3G speeds on EE, Three and Vodafone. 3% 3% 3% 4% 3% 14% 6% 28% 14% 10% 47% 39% 45% 52% 52% 18% 23% 16% 18% 18% 17% 30% 9% 12% 18% 0% 20% 40% 60% 80% 100% All EE O2 Vodafone Three Failed <2Mbit/s 2-15Mbit/s 15-25Mbit/s >25Mbit/s Distributions show consistency 15 Figure 6: Average download speed – 4G versus 3G, by operator Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered on 4G only and 3G only. BBC homepage loading time: results For our web browsing test, we measured the time taken to fully download the BBC homepage on each network. This is the time taken for the website to load all the content contained on the homepage, rather than the time until a consumer can interact with the page. Overall, EE had the quickest BBC web browsing time, at five seconds, compared to six seconds for all of the other operators. The longest average web browsing time in three of the five cities was on O2’s network. Across all five cities, the average web browsing time was six seconds. Figure 7: Average 4G tariff BBC homepage load time, by city Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. 17 22 13 15 17 6 7 5 7 7 0 5 10 15 20 25 All EE O2 Vodafone Three 4G 3G Mbit/s 3G significant differences =1st 4th =1st =1st 6 6 6 6 6 6 5 5 6 5 5 6 6 7 6 6 7 6 6 6 6 5 6 6 6 5 6 6 6 6 0 2 4 6 8 Overall Cardiff Edinburgh Liverpool London Norwich All EE O2 Vodafone Three seconds The shortest time is best 3G significant difference rankings. 16 Figure 8: Average 4G tariff BBC homepage load time: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. The highest proportion of successful web browsing attempts occurred on EE’s network, on which 98% of attempts were successful. Three’s network had the second greatest success rate, with 95% of web browsing attempts succeeding. The operator with the highest proportion of web browsing speeds under 4.5 seconds was EE, with a quarter of samples achieving this speed (25%), followed by Three (15%), Vodafone (9%) and O2 (6%). Figure 9: Distribution of 4G tariff BBC homepage loading time results Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. The variations in distributions across each city are summarised in the executive summary; more in-depth figures are provided in the chart pack accompanying this publication. The results of the BBC web browser tests should be read in line with the standard web page results, which show how different aspects of a network come into play when different content is loaded. Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall EE Three Vodafone O2 Cardiff EE, Three (=) - Vodafone O2 Edinburgh EE, O2 (=) - Three, Vodafone (=) - Liverpool EE Three, Vodafone (=) - O2 London EE Three Vodafone O2 Norwich EE, Three (=) - O2, Vodafone (=) - 14% 25% 6% 10% 15% 67% 66% 65% 68% 69% 12% 7% 18% 13% 11% 7% 2% 11% 9% 5% 0% 20% 40% 60% 80% 100% All EE O2 Vodafone Three <4.5s 4.5-7s >7s Failed Proportion of samples Remember – you may not notice these differences 17 3G versus 4G: BBC homepage load time results The improvements potentially available to consumers if they were to upgrade to 4G from 3G are also seen with the BBC homepage load times. Across each network, it took less time to load the homepage in full when using a 4G network than when using a 3G network. Figure 10: Average BBC home page load time, 3G versus 4G Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: average (mean) of all tests gathered on 4G only and 3G only. 6 5 6 6 5 7 7 7 7 7 0 2 4 6 8 All EE O2 Vodafone Three 4G 3G seconds 3G significant differences 4th =1st =1st =1st 3G significant difference rankings 18 Results: internet activities Internet activities: results This section presents the results of a range of tests which relate to common internet activities. More detail on these tests, and city-by-city analysis, can be found in the technical annex and metric summary table. Upload speed refers to the speed at which information is transferred from a device to the internet. For example, to post a photo taken on a smartphone to Facebook, a consumer will upload the photo to Facebook from their device. YouTube streaming is a process in which short-form video content is viewed in real time. This metric indicates the proportion of videos streamed on each network that were in standard definition, high definition, and what proportion failed to play. Response time is the delay between a consumer making a request to their mobile network for information and the network providing this information to the device. A quick response time is important for activities that require information to be delivered with a little delay as possible. In particular, the responsiveness of a network is significant when using Skype or FaceTime. Standard web page load time refers to how long it took for a basic web page to load on each of the operators’ networks, from the time between a consumer sending a request for a web page and the page finishing loading. For this test we used an industry standard-sized web page (based on an ETSI ‘mKepler’ standard reference page) which is static – its content does not change. Comparisons with these standard web page results and our BBC browser results show how different aspects of mobile networks come into play for different tasks. Upload speed Across all the five cities, EE achieved the fastest upload speeds, averaging 20Mbit/s. The operators which achieved the lowest average speed overall were O2 and Vodafone (10Mbit/s). Upload speeds were slowest in Norwich, which had an average speed of 11Mbit/s across all operators. 19 Figure 11: Average 4G tariff upload speed, by city Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. Figure 12: Average 4G tariff upload speed: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. YouTube streaming We used the trailer for the latest Bond film, Spectre, in our YouTube streaming test. When loading this video, the greatest proportion of high-definition video streams were on EE’s network at 97% followed by Three (87%) and Vodafone at 87% and 86% respectively across all five cities. The video failed to load at all on 7% of occasions on O2, while EE had the lowest failure rate, at 1%. 13 12 15 14 12 11 20 21 21 20 21 15 10 8 13 11 7 8 10 9 13 12 10 7 12 12 1211 10 14 0 5 10 15 20 25 Overall Cardiff Edinburgh Liverpool London Norwich All EE O2 Vodafone Three Mbit/s Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall EE Three Vodafone O2 Cardiff EE Three Vodafone O2 Edinburgh EE O2, Vodafone (=) - Three Liverpool EE O2, Three, Vodafone (=) - - London EE Three, Vodafone (=) - O2 Norwich EE, Three (=) - O2 Vodafone 20 Figure 13: Video streaming resolution on a 4G tariff Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Proportion of all tests gathered with 4G as the best bearer. HD videos here have a resolution of 720p and SD videos have a resolution of 360p. Standard web page loading time: results In addition to our BBC homepage web browsing test, we tested web browsing performance on a standard web page containing a small amount of static content. The average time taken to load a standard static web page, on each of the operators’ networks, was around one second. The slowest standard web page loading time was two seconds, on Three’s network in Cardiff and Edinburgh, and on Vodafone’s network in London. Figure 14: Average 4G tariff standard web page load time Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (median) of all tests gathered, with 4G as the best bearer. 89% 97% 85% 86% 87% 7% 2% 8% 10% 9% 4% 7% 4% 3% 0% 20% 40% 60% 80% 100% All EE O2 Vodafone Three Failed SD HD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 2 2 1 1 1 0 1 2 Overall Cardiff Edinburgh Liverpool London Norwich All EE O2 Vodafone Three seconds A ‘standard web page’ is a very simple page 21 Figure 15: Average 4G tariff standard web page load time: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. It is worth noting that the results of the standard web page test differ from those of the BBC web page test. This highlights the difference between the two sites, e.g. the larger, dynamic BBC homepage compared to the static content on the standard web page, and how networks respond differently depending on the request being made of it. Response time On average, Three had the most responsive network, taking 44ms for a packet of data to get to a consumer’s device following the request. The network with the longest response time overall was O2, at 92ms. The response times of networks were closest in Norwich, where the range of response times was only 18ms. Figure 16: Response 4G tariff response time Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (median) of all 32 byte ping tests gathered, with 4G as the best bearer. Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall EE, O2 (=) - Vodafone Three Cardiff EE O2 Vodafone Three Edinburgh EE O2 Vodafone Three Liverpool Vodafone EE, O2 (=) - Three London O2 EE Three Vodafone Norwich O2 EE Three Vodafone 63 64 79 66 59 58 56 49 65 55 46 64 92 104 93 110 82 66 81 85 90 84 78 59 44 43 47 41 42 48 0 20 40 60 80 100 120 Overall Cardiff Edinburgh Liverpool London Norwich All EE O2 Vodafone Three ms The shortest response time is best 22 Figure 17: Response time on a 4G tariff: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. Call success rate Successfully setting up and maintaining a voice call remains important for many consumers using their smartphones. The vast majority of calls were successful throughout our test period, with 99% of calls being completed on EE and 98% on O2, Three and Vodafone. Figure 18: Call success rate Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall Three EE Vodafone O2 Cardiff Three EE Vodafone O2 Edinburgh Three EE Vodafone O2 Liverpool Three EE Vodafone O2 London Three EE Vodafone O2 Norwich Three Vodafone EE O2 98% 99% 98% 98% 98% 0% 20% 40% 60% 80% 100% All EE O2 Vodafone Three Completed Dropped Failed Proportion of samples 23 Time-of-day results Analysing results across the day gives an important reflection of mobile network performance, as it allows us to look at how the networks respond under stressed conditions and when load is at its highest (which, by definition, is when the most consumers are trying to use services). The following charts split out the average download speeds, and BBC homepage load times, for each operator, across all five cities on weekday mornings (7am to 10am), at midday (10am – 4.30pm), in the evening (4.30pm – 8pm) and at weekends (7am – 8pm). Download speeds Overall, the fastest download speeds were seen in the mornings (Mon-Fri) and at the weekends, achieving speeds of 16 Mbit/s and 15Mbit/s respectively. Weekday evenings provided the slowest average download speeds for each operator. These results show significant drop-offs in download speeds for O2, Vodafone and Three across weekdays, but EE was able to maintain a similar level of performance across all of the day-parts that we tested. Figure 19: Average 4G tariff download speed, by MNO and time of day Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. Morning is 7am to 10am, midday is 10am to 4.30pm and evening is 4.30pm to 8pm. 16 21 11 13 18 14 19 9 12 15 11 20 6 9 9 15 22 12 13 14 0 5 10 15 20 25 All EE O2 Vodafone Three Morning (Mon-Fri) Midday (Mon-Fri) Evening (Mon-Fri) Weekend Mbit/s 24 Figure 20: Average 4G download tariff speed, by MNO and time of day: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. Morning is 7am to 10am, midday is 10am to 4.30pm and evening is 4.30pm to 8pm (all Monday to Friday). The BBC homepage results tell a similar story regarding consistency of network performance. Load times were slowest across weekday evenings for each operator, although there was no significant difference between the evenings and weekends for EE. Figure 21: Average 4G tariff BBC homepage load time, by MNO and time of day Source: Ofcom mobile broadband measurements, fieldwork November and December 2015. Note: Average (mean) of all tests gathered, with 4G as the best bearer. Morning is 7am to 10am, midday is 10am to 4.30pm and evening is 4.30pm to 8pm. Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall Morning, Weekends (=) - Midday Evening EE Morning, Weekends (=) - Midday, Evening (=) - O2 Weekends Morning Midday Evening Vodafone Afternoon, Morning, Weekends (=) - - Evening Three Morning Midday, Weekends (=) - Evening 6 5 6 6 6 6 5 6 6 6 6 6 7 6 6 6 6 6 6 6 0 2 4 6 8 All EE O2 Vodafone Three Morning (Mon-Fri) Midday (Mon-Fri) Evening (Mon-Fri) Weekend seconds 25 Figure 22: Average 4G tariff BBC browser time, by MNO and time of day: significant differences Source: Ofcom Note: Differences are significant to a 95% confidence level. Morning is 7am to 10am, midday is 10am to 4.30pm and evening is 4.30pm to 8pm (all Monday to Friday). Fastest 2nd Fastest 3rd Fastest 4th Fastest Overall Morning, Midday (=) - Weekend Evening EE Morning Midday Weekend, Evening (=) - O2 Morning, Midday, Weekend (=) - - Evening Vodafone Morning, Midday, Weekend (=) - - Evening Three Morning, Midday (=) - Weekend Evening 26 Metrics tables Figure 23: Results snapshot: Cardiff and Edinburgh Source: Ofcom. Note: ‘s’ is seconds and ‘ms’ is milliseconds. ‘MOS’ stands for mean opinion score and scores range from 1 (bad) to 5 (excellent). Cardiff Edinburgh Metric EE O2 Three Vodafone EE O2 Three Vodafone Download Speed Average speed 23 Mbit/s 6 Mbit/s 14 Mbit/s 9 Mbit/s 21Mbit/s 15Mbit/s 15Mbit/s 15Mbit/s % over 2Mbit/s 95% 54% 92% 74% 92% 96% 87% 94% Web browsing BBC homepage load time: average 5s 7s 5s 6s 6s 6s 6s 6s BBC homepage success rate 99% 88% 97% 89% 96% 92% 96% 92% Standard webpage load time: average 1s 1s 2s 1s 1s 1s 2s 1s Standard webpage success rate 99% 95% 98% 90% 96% 96% 95% 96% Upload speed: average 21Mbit/s 8Mbit/s 12Mbit/s 9Mbit/s 21Mbit/s 13Mbit/s 12Mbit/s 13Mbit/s YouTube % of HD streams 98% 83% 94% 88% 96% 96% 72% 91% Time to first picture: average 2s 4s 2s 3s 2s 3s 2s 3s Video Quality: average MOS 4 4 4 4 4 4 4 4 Response time: average 49ms 104ms 43ms 85ms 65ms 93ms 47ms 90ms Voice calls Mobile to fixed call success rate 99% 97% 99% 98% 98% 97% 98% 98% Mobile to mobile call quality: average MOS 4 4 4 4 4 4 4 4 27 Figure 24: Results snapshot: Liverpool, London and Norwich Source: Ofcom. Note: ‘s’ is seconds and ‘ms’ is milliseconds. ‘MOS’ stands for mean opinion score and scores range from 1 (bad) to 5 (excellent) Liverpool London Norwich Metric EE O2 Three Vodafone EE O2 Three Vodafone EE O2 Three Vodafone Download Speed Average speed 20Mbit/s 8Mbit/s 15Mbit/s 14Mbit/s 17Mbit/s 7Mbit/s 9Mbit/s 13Mbit/s 19Mbit/s 13Mbit/s 22Mbit/s 10Mbit/s % over 2Mbit/s 91% 73% 89% 93% 92% 59% 81% 80% 89% 67% 88% 70% Web browsing BBC homepage load time: average 5s 6s 6s 5s 5s 7s 6s 6s 6s 6s 6s 6s BBC homepage success rate 99% 92% 97% 97% 99% 87% 96% 87% 96% 85% 90% 90% Standard webpage load time: average 1s 1s 1s 1s 1s 1s 1s 2s 1s 1s 1s 1s Standard webpage success rate 99% 97% 95% 99% 99% 97% 98% 92% 99% 91% 93% 95%