Google's Tensor inside of Pixel 6, Pixel 6 Pro: A Look into Performance & Efficiency
by Andrei Frumusanu on November 2, 2021 8:00 AM EST- Posted in
- Mobile
- Smartphones
- SoCs
- Pixel 6
- Pixel 6 Pro
- Google Tensor
Google's IP: Tensor TPU/NPU
At the heart of the Google Tensor, we find the TPU which actually gives the chip is marketing name. Developed by Google with input and feedback by the team’s research teams, taking advantage of years of extensive experience in the field of machine learning, Google puts a lot of value into the experiences that the new TPU allows for Pixel 6 phones. There’s a lot to talk about here, but let’s first try to break down some numbers, to try to see where the performance of the Tensor ends up relative to the competition.
We start off with MLCommon’s MLPerf – the benchmark suite works closely with all industry vendors in designing something that is representative of actual workloads that run on devices. We also run variants of the benchmark which are able to take advantage of various vendors SDKs and acceleration frameworks. Google had sent us a variant of the MLPerf app to test the Pixel 6 phones with – it’s to be noted that the workloads on the Tensor run via NNAPI, while other phones are optimised to run through the respective chip vendor’s libraries, such as Qualcomm’s SNPE, Samsung’s EDEN, or MediaTek’s Neuron – unfortunately only the Apple variant is lacking CoreML acceleration, thus we should expect lower scores on the A15.
Starting off with the Image Classification, Object Detection, and Image Segmentation workloads, the Pixel 6 Pro and the Google Tensor showcase good performance, and the phone is able to outperform the Exynos 2100’s NPU and software stack. More recently, Qualcomm had optimised its software implementation for MLPerf 1.1, able to achieve higher scores than a few months ago, and this allows the Snapdragon 888 to achieve significantly better scores than what we’re seeing on the Google Tensor and the TPU – at least for those workloads, in the current software releases and optimisations.
The Language Processing test of MLPerf is a MobileBERT model, and here for either architectural reasons of the TPU, or just a vastly superior software implementation, the Google Tensor is able to obliterate the competition in terms of inference speed.
In Google’s marketing, language processing, such as live transcribing, and live translations, are very major parts of the differentiating features that the new Google Tensor enables for the Pixel 6 series devices – in fact, when talking about the TPU performance, it’s exactly these workloads that the company highlights as being the killer use-cases and what the company calls state-of-the-art.
If the scores here are indeed a direct representation of Google’s design focus of the TPU, then that’s a massively impressive competitive advantage over other platforms, as it represents a giant leap in performance.
Other benchmarks we have available are for example GeekBench ML, which is currently still in a pre-release state in that the models and acceleration can still change in further updates.
The performance here depends on the APIs used, with the test either allowing TensorFlow delegates for the GPU or CPU, or using NNAPI on Android devices (and CoreML on iOS). The GPU results should only represent the GPU ML performance, which is surprisingly not that great on the Tensor, as it somehow lands below the Exynos 2100’s GPU.
In NNAPI mode, the Tensor is able to more clearly distinguish itself from the other SoCs, showcasing a 44% lead over the Snapdragon 888. It’s likely this represent the TPU performance lead, however it’s very hard to come to conclusions when it comes to such abstractions layer APIs.
In AI Benchmark 4, when running the benchmark in pure NNAPI mode, the Google Tensor again showcases a very large performance advantage over the competition. Again, it’s hard to come to conclusions as to what’s driving the performance here as there’s use of CPU, GPU, and NPUs.
I briefly looked at the power profile of the Pixel 6 Pro when running the test, and it showcased similar power figures to the Exynos 2100, which extremely high burst power figures of up to 14W when doing individual inferences. Due to the much higher performance the Tensor showcases, it also means it’s that much more efficient. The Snapdragon 888 peaked around 12W in the same workloads, so the efficiency gap here isn’t as large, however it’s still in favour of Google’s chip.
All in all, Google’s ML performance of the Tensor has been its main marketing point, and Google doesn’t disappoint in that regard, as the chip and the TPU seemingly are able to showcase extremely large performance advantages over the competition. While power is still very high, completing an inference faster means that energy efficiency is also much better.
I asked Google what their plans are in regards to the software side of things for the TPU – whether they’ll be releasing a public SDK for developers to tap into the TPU, or whether things will remain more NNAPI centric like how they are today on the Pixels. The company wouldn’t commit yet to any plans as it’s still very early – in generally that’s the same tone we’ve heard from other companies as even Samsung, even 2 years after the release of their first-gen NPU, doesn’t publicly make available their Eden SDK. Google notes that there is massive performance potential for the TPU and that the Pixel 6 phones are able to use them in first-party software, which enables the many ML features for the camera, and many translation features on the phone.
Facebook
Twitter
RSS
108 Comments
View All Comments
sharath.naik - Thursday, November 4, 2021 - link
Good in-depth review. I know you are doing the camera review of this. So I have a request can you look into if the Pixel 6 cameras are hardware binned to ~12MP even though the specs say they are 50MP/48MP. There is a lot of mixed views out there, most mistaking this as the pixel binning done on other phones like galaxy S21u(Software binned for low light but has access to full resolution). If you could confirm this for the review would be great, looking forward to that review.Silver5urfer - Tuesday, November 2, 2021 - link
Exactly as expected 1:1The SoC is a bust, they tried to do some gimmickry with their zero talent and tried to make it a cheaper deal by going to Samsung for their IP fabrication expertise and lithography process. Ended up being a dud in CPU, GPU and price to performance, all that NPU NN, mega magic boom is all a farce. I was asking the same thing, what does these provide to us end users ? Nothing. Just that fancy Livetranslation and other gimmicks which you use seldom. On top we do not even know what TPU does in the Pixel Software, it's closed source. AOSP is open but Pixel UI and all backend are closed.
Hardware is utter joke, the P6 series has garbage mmwave system look at the internals, they crammed one lol. LG V50 back in 2019 had 2-33 mmwave antennas. This junk saved on cost. The display has banding issues all over the place. Optical image sensor for Fingerprint is slow and a joke vs the physical dedicated ones. The stereo speaker system has massive channel imbalance on top. Then you have the low battery SoT for this price point and battery capacity. The DIY aspect is thrown into gutters, the phone has massively hamfisted cooling approach with graphite pads smeared all over the place as leaks showed and no proper HS system it's just a small pathetic AL board reinforcement plate doing it so on top the Display has no metal backplate to reinforce it or dissipate heat. What a nonsense. SD888 itself heats up a lot and so many vendors add VC cooling, Sony Xperia 1 Mark 3 messed up there and had inferior performance with throttling. This junk is even more pathetic, pay for a S tier SKU get trash sustained performance of a B+ device, the AP, Pocket now and other Youtube shill press will hype this to moon.
We do not even know how this junk has in terms of Software blocks like P2, P3, P4 had A/B system, then merged partitions, later read only ext4 system. This will have even worse. To make it a round about trash, the software is a joke cheap kiddo inspired garbage, heck that BBKs OnePlus's Oxygen + Oppo Color OS mix is way superior than this junk with massive information density loss.
I'd wait for the next SD successor device, hopefully 888's BS power consumption and insane heat can be reduced.
Silver5urfer - Tuesday, November 2, 2021 - link
It's a typo for mmwave, it's 2-3 units. Also I forgot to mention the lack of charger, SD slot, no 3.5mm jack very poor servicing almost impossible to get the phone properly cooled if you open it due to cheap graphite pad reliance. It also has that USB port and microphone soldered to the mainboard which looks like a feeble trash unit check any phone in the recent times and look how solid engineered they are, dual sandwich designs with reinforced chassis and proper heat dissipation.goatfajitas - Tuesday, November 2, 2021 - link
People get WAY too hung up on benchmarks. LOL, a "dud". A phone is about user experience, not how many "geekmarks" = best.lionking80 - Tuesday, November 2, 2021 - link
I agree that the benchmarks do not tell the whole story, but I would still say that even the use of a Snapdragon 870 would have been a better choice.The general performance is similar (maybe a small advantage for Tensor in AI), but the advantages of Snapdragon 870 are bigger: runs much cooler, hugely better battery-life.
To be honest I am disappointed by the SOC. The only thing that might make it a seller is the software (ui and camera), but the SOC is rather a no-go.
goatfajitas - Tuesday, November 2, 2021 - link
There are other factors though. Early ROM settings, tweaks, bugs, and cooling/hardware. The 870 may have scored lower in a P6 as well. So many factors. - Agreed, the P6 should be a bit more polished though.at_clucks - Tuesday, November 2, 2021 - link
The problem goes beyond the slightly worse SoC than the already existing Qualcomm offering. It's that despite being a "Google SoC" they still support it for just 3 years. All the excuses used over the years, all the pointing fingers at SoC manufacturers for the lack of support were just proven to be a load of crap. Same crap, now with a google sticker.sharath.naik - Tuesday, November 2, 2021 - link
It's about to get worse with the camera review. I can verify Google might have been bluffing about the 50mp/48mp sensors. The sensors are locked at 12mp. So Pixel pro has essentially three 12 mp cameras. Which means the portrait mode zoom of 2x is a low resolution 3 MP image. Also at 10x zoom the image resolution is 2.5MP, 4 times lower than that of s21 ultra. What drove Google to make the choice of first hardware pixel binning the resolution down and then trying to digitally blow the resolution backup!!.It's baffling, tried to get an answer from Google support, they just refused to confirm or deny this is binned at the hardware levelhoxha_red - Tuesday, November 2, 2021 - link
"I can verify that google might have been bluffing"dude, lmfao—it's called "binning"; please look it up. they've been upfront about this and it was known before the phone was even launched, let alone after we've seen all of these reviews. The reason Google support "refused to confirm or deny" is because the people doing customer support are unlikely to know what "pixel binning" is (hey, I guess they're in good company there with you), and are not equipped to deal with weirdos of your specific variety.
Maxpower27 - Tuesday, November 2, 2021 - link
You obviously have no familiarity with mobile phone cameras and sensors in particular. Read up about them and then try again.