AI

Microsoft wants to make Windows an AI operating system, launches Copilot+ PCs

Comment

A Microsoft logo is displayed at the MWC (Mobile World Congress) in Barcelona on March 2, 2022. - The Mobile World Congress, where smartphone and telecoms companies show off their latest products and reveal their strategic visions, is expected to welcome more than 40,000 guests over its four-day run. (Photo by Josep LAGO / AFP) (Photo by JOSEP LAGO/AFP via Getty Images)
Image Credits: JOSEP LAGO / Getty Images

Microsoft wants to bring generative AI to the forefront of Windows — and the PCs running it.

At a pair of keynotes during its annual Build developer conference this week, the company unveiled a new lineup of Windows machines it’s calling Copilot+ PCs, plus generative AI-powered features like Recall, which helps users find apps, files and other content they’ve viewed in the past. Copilot, Microsoft’s brand of generative AI, will soon be far more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.

We’ve rounded up all the major announcements from Monday and Tuesday here.

Volumetric Apps

Microsoft is bringing Windows Volumetric Apps — basically spatially aware, interactive VR apps — to Meta Quest headsets. Through a partnership with Meta, Microsoft says that it’ll deliver Windows 365 and local PC connectivity to Quest headsets, enabling developers to extend their apps into 3D space.

Image Credits: Microsoft

During Tuesday’s keynote, Microsoft showed off a digital exploded 3D view of an Xbox controller from the perspective of a Meta Quest 3 headset — a digital object the wearer could manipulate with their hands. “We’re deepening our partnership with Meta to make Windows a first-class experience on Quest devices,” Pavan Davuluri, CVP of Windows and devices at Microsoft, said during the demo.

Developers can sign up for a preview to receive access to Microsoft’s new volumetric API.

Copilot+ PCs

Microsoft Copilot+ PCs
Image Credits: Microsoft

Copilot+ PCs are Microsoft’s vision of AI-first, flagship Windows hardware. All include dedicated chips called NPUs to power AI experiences like Recall. And they ship with 16GB of RAM minimum, paired with SSD storage.

The first Copilot+ PCs will pack Qualcomm’s Snapdragon X Elite and Plus chips, which Microsoft claims delivers up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also committed to building processors for Copilot+ devices in partnership with a range of manufacturers, including Acer, Asus, Dell, HP, Lenovo and Samsung.

Copilot+ PCs start at $999, and some are available for preorder today.

Surface Pro and Surface Laptop

Microsoft’s newly unveiled Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery.

Microsoft Surface Laptop
Image Credits: Microsoft

The latest Surface Laptop — available with a 13.8- or 15-inch display — has been redesigned with “modern lines” and thinner screen bezels. It lasts up to 22 hours on a charge and is up to 86% faster than the Surface Laptop 5, the company says. It also supports Wi-Fi 7 and has a haptic feedback touchpad.

Microsoft Surface Pro
Image Credits: Microsoft

As for the new Surface Pro, Microsoft says it’s up to 90% faster than the previous-gen Surface Pro (the Surface Pro 9), and it sports a new OLED with HDR display, Wi-Fi 7 (and optional 5G) and an upgraded ultrawide front-facing camera. Also, its detachable keyboard — which has been reinforced with additional carbon fiber — now has haptic feedback.

Recall

Windows 11’s forthcoming Recall feature can “remember” apps and content a user accessed on their PC weeks or even months ago, for example helping them to find a Discord chat where they were discussing clothes they were considering buying. Users can use Recall’s timeline to “scroll back” to see what they were working on in the recent past and drill down in files like PowerPoint presentations to surface info potentially relevant to their searches.

Microsoft says that Recall can create associations between colors, images and more to let users search for practically anything on their PCs in natural language (not dissimilar to startup Rewind’s tech); developers will be able to improve Recall by adding contextual information to their apps. And Microsoft claims all user data associated with Recall is kept private and on-device — and not used to train AI models, importantly.

Here’s more from Microsoft: “Your snapshots are yours; they stay locally on your PC. You can delete individual snapshots, adjust and delete ranges of time in Settings, or pause at any point right from the icon in the System Tray on your Taskbar. You can also filter apps and websites from ever being saved.”

Image editing and live translations

There’s now more AI in Windows than ever and some of it exclusively on the new Copilot+ PCs.

A new feature called Super Resolution can restore old photos by automatically upscaling them. And Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can generate images and also ask the AI model to follow what they’re drawing to change or restyle the image.

Live Captions with live translation
Image Credits: Microsoft

Elsewhere, Live Captions with live translations translates any audio that passes through a PC — whether from YouTube or a local file — into the language of the user’s choosing. Live translations will initially support around 40 languages, including English, Spanish, Mandarin and Russian.

A separate but related new feature in Microsoft Edge offers real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg and more. Set to become available in the near future, the feature — which supports the translation of Spanish into English and English to German, Hindi, Italian, Russian and Spanish — translates spoken content through both dubbing and subtitles live.

Team Copilot and extensions

Team Copilot is the latest expansion of Microsoft’s growing Copilot suite of generative AI tech. It integrates with Teams, the company’s videoconferencing app, to help manage meeting agendas and take notes that anyone in a meeting can co-author. And it extends to Loop and Planner, Microsoft’s collaboration and planning platforms, to create and assign tasks, track deadlines and notify team members when their input’s needed.

Image Credits: GitHub/Microsoft

In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allow developers to extend GitHub’s code-generating tool GitHub Copilot with third-party apps and skills. Launch partners include DataStax, Docker and LambdaTest; extensions will live in the GitHub Marketplace, but developers will also be able to create their own private extensions to integrate with their internal systems and APIs.

Windows Copilot Runtime

Microsoft Windows Copilot Runtime
Image Credits: Microsoft

Powering capabilities such as Recall and Super Resolution is the Windows Copilot Runtime, a collection of ~40 generative AI models that make up what Microsoft describes as “a new layer” of Windows. In tandem with the semantic index, a vector-based system local to an individual Copilot+ PC, the Windows Copilot Runtime allows generative AI-powered apps — including third-party apps — to run without necessarily needing an internet connection.

“[The runtime] consists of ready-to-use AI APIs like Studio Effects, Live Captions translations, OCR, Recall with user activity and [more], which will be available to developers in June,” Davuluri said on Tuesday.

Microsoft says that CapCut, the popular video editor from TikTok owner ByteDance, will use the Windows Copilot Runtime and accompanying new Windows Copilot Library, a set of APIs and AI dev tools, to speed up its AI features. And Meta will add the aforementioned Studio Effects to WhatsApp to deliver features such as background blur and eye contact during video calls.

Upgraded bot builders

Azure AI Studio, the toolset within Microsoft’s Azure OpenAI Service that lets customers combine an AI model and build an app that “reasons over” that data, will soon allow developers to create apps using pay-as-you-go inference APIs — the APIs through which developers can access and fine-tune generative AI models hosted on Azure infrastructure. Microsoft calls this “model-as-a-service,” and it’s launching with models from Nixtla and Core42 to start.

In the adjacent Copilot Studio product suite, Microsoft is launching Copilot agents, which the company describes as AI bots that can “independently orchestrate tasks tailored to specific roles and functions.” (Copilot Studio provides tools to connect Copilot for Microsoft 365, the AI-powered “copilot” in apps like Excel and Word, to third-party data.) Leveraging memory and knowledge of context, Copilot agents can navigate various types of business workflows, learning from user feedback and asking for help when they encounter situations they don’t know how to handle.

Snapdragon Dev Kit

Qualcomm dev kit
Image Credits: Microsoft

There’s a new dev kit from Qualcomm aimed at developers building apps for Arm-chip-packing Copilot+ PCs

The $899.99 Snapdragon Dev Kit for Windows — which measures around the same width, height and length as Apple’s Mac Mini, incidentally — houses Qualcomm’s Snapdragon X Elite chip paired with 32GB of RAM, 512GB of storage and lots of I/O. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4, and, via its various USB-C and HDMI ports, it can drive up to three 4K monitors at once.

Phi-3

Microsoft announced an addition to its generative AI model family Phi, Phi-3-vision, that can do general visual analysis and reasoning tasks, such as answering questions about charts and images. The model can read both text and images and is efficient enough to run on a mobile device.

Phi-3-vision is available in preview, while the model’s previously announced text-only counterparts — Phi-3-mini, Phi-3-small and Phi-3-medium — are now generally available.

Partnership with Khan Academy

Microsoft is teaming up with Khan Academy to donate access to cloud compute infrastructure, allowing Khan Academy to offer educators in the U.S. free access to Khan Academy’s AI-powered tools. The two companies will also collaborate to explore opportunities to improve AI apps for math tutoring through generative AI, Microsoft said on Tuesday.

We’re launching an AI newsletter! Sign up here to start receiving it in your inboxes on June 5.

More TechCrunch

Ola Electric, India’s largest electric two-wheeler maker, saw its shares rise as much as 20% on its public debut on Friday, making it the biggest listing among Indian firms in…

Ola Electric surges in India’s biggest listing in two years

Rocket Lab surpassed $100 million in quarterly revenue for the first time, a 71% increase from the same quarter of last year. This is just one of several shiny accomplishments…

Rocket Lab’s sunny outlook bodes well for future constellation plans 

In 1996, two companies, Patersons HR and Payroll Solutions, formed a venture called CloudPay to provide payroll and payments services to enterprise clients. CloudPay grew quietly over the next several…

CloudPay, a payroll services provider, lands $120M in new funding

The vulnerabilities allowed one security researcher to peek inside the leak sites without having to log in.

Security bugs in ransomware leak sites helped save six companies from paying hefty ransoms

Featured Article

A comprehensive list of 2024 tech layoffs

The tech layoff wave is still going strong in 2024. Following significant workforce reductions in 2022 and 2023, this year has already seen 60,000 job cuts across 254 companies, according to independent layoffs tracker Layoffs.fyi. Companies like Tesla, Amazon, Google, TikTok, Snap and Microsoft have conducted sizable layoffs in the…

A comprehensive list of 2024 tech layoffs

A new “beta rabbit” mode adds some conversational AI chops to the Rabbit r1, particularly in more complex or multi-step instructions.

Rabbit’s r1 refines chats and timers, but its app-using ‘action model’ is still MIA

Los Angeles is notorious for its back-to-back traffic. Three events that promise to bring in millions of spectators from around the world — the 2026 World Cup, the Super Bowl…

Archer to set up air taxi network in LA by 2026 ahead of World Cup

Featured Article

Amazon is fumbling in India

Amazon’s decision to overlook quick-commerce in India is now looking like a significant misstep.

Amazon is fumbling in India

OpenAI’s GPT-4o, the generative AI model that powers the recently launched alpha of Advanced Voice Mode in ChatGPT, is the company’s first trained on voice as well as text and…

OpenAI finds that GPT-4o does some truly bizarre stuff sometimes

On Thursday, Box filled in a missing piece on its AI platform when it bought automated metadata extracting startup, Alphamoon.

Box adds crucial piece to its AI platform with Alphamoon acquisition

OpenAI has announced a new appointment to its board of directors: Zico Kolter. Kolter, a professor and director of the machine learning department at Carnegie Mellon, predominantly focuses his research…

OpenAI adds a Carnegie Mellon professor to its board of directors

Count Spotify and Epic Games among the Apple critics who are not happy with the iPhone maker’s newly revised compliance plan for the European Union’s Digital Markets Act (DMA). Shortly…

Spotify and Epic Games call Apple’s revised DMA compliance plan ‘confusing,’ ‘illegal’ and ‘unacceptable’

Thursday seeks to shake up conventional online dating in a crowded market. The app, which recently expanded to San Francisco, fosters intentional dating by restricting user access to Thursdays. At…

Thursday, the dating app that you can use only on Thursdays, expands to San Francisco

AI companies are gobbling up investor money and securing sky-high valuations early in their life cycle. This dynamic has many calling the AI industry a bubble. Nick Frosst, a co-founder…

Cohere co-founder Nick Frosst thinks everyone needs to be more realistic about what AI can and cannot do

Instagram is rolling out the ability for users to add up to 20 photos or videos to their feed carousels, as the platform embraces the trend of “photo dumps.” Back…

Instagram is embracing the ‘photo dump’

Welcome back to TechCrunch Mobility — your central hub for news and insights on the future of transportation. Sign up here for free — just click TechCrunch Mobility! Anyone paying…

Lyft ‘opens a can of whoop ass’ on surge pricing, Tesla’s Dojo explained and Saudi Arabia pumps $1.5B into Lucid

Flint Capital just closed its third fund at $160 million. Its has a unique strategy for finding its limited partner investors. 

Flint Capital raises a $160M through an unusual fund-raising strategy

Earlier this week it emerged that the DPC had instigated court proceedings seeking an injunction against X over the data processing without consent.

Elon Musk’s X agrees to pause EU data processing for training Grok

During testing, Google DeepMind’s table tennis bot was able to beat all of the beginner-level players it faced.

Google DeepMind develops a ‘solidly amateur’ table tennis robot

The X account announced that its Premium+ subscription would now be “fully” ad-free, leading some to question how this change would affect creator earnings.

As X sues advertisers over boycott, the app ditches all ads from its top subscription tier

Apple has further revised its compliance plan for the European Union’s Digital Markets Act (DMA) rulebook, which, since March, has forced it to give iOS developers more freedom over how…

Apple revises DMA compliance for App Store link-outs, applying fewer restrictions and a new fee structure

The rise of neobanks has been fascinating to witness, as a number of companies in recent years have grown from merely challenging traditional banks to being massive players in and…

Chime and Dave execs are coming to TechCrunch Disrupt 2024

If you visited the Wikipedia website on mobile this week, you might have seen a pop-up indicating that dark mode is ready for prime time.

How to enable Wikipedia’s dark mode

The home security company says attackers accessed databases containing customer home addresses, email addresses, and phone numbers.

Home security giant ADT says it was hacked

The Looking Glass Pro has a 6-inch display and a foldable base. It shows spatial images like those created with the Apple Vision Pro and iPhone 15 Pro.

Looking Glass’ new lineup includes a $300 phone-sized holographic display

TikTok’s latest offering is capitalizing on the app’s ability to serve as a discovery engine for other media — something its users already take advantage of by sharing short clips…

TikTok partners with Warner Bros. to become a discovery engine for TV and movies

Cocoon is a new startup built on the belief that greener steel production and the creation of concrete slag doesn’t have to be an either/or proposition.

Cocoon is transforming steel production runoff into a greener cement alternative

SoundHound, an AI company that makes voice interface tech used by car companies, restaurants and tech firms, is doubling down on enterprise services by playing consolidator in a crowded market.…

SoundHound acquires Amelia AI for $80M after it raised $189M+

Seeking mental health support is a complex process, but some founders believe that using AI to formalize techniques like cognitive behavioral therapy (CBT) can help folks who might not have…

Feeling Great’s new therapy app translates its psychiatrist co-founder’s experience into AI

The U.K.’s antitrust regulator has confirmed that it’s carrying out a formal antitrust investigation into Amazon’s ties with Anthropic, after Amazon recently completed a $4 billion investment into the AI startup.…

UK launches formal probe into Amazon’s ties with AI startup Anthropic