All the big news from Apple’s latest keynote presentation
Apple’s annual developer keynote just concluded, and though it lacked an earth-shattering reveal on the level of last year’s Vision Pro announcement, the presentation was nevertheless packed with some pretty major reveals.
As usual, there were previews of all Apple’s new software, including some pretty nifty additions in iOS 18 and iPadOS 18. But stealing the show was Apple’s long-awaited push into AI, rebranded as “Apple Intelligence” and sitting at the heart of almost everything else we saw.
If you missed the livestream, we’ve got you covered. Below is everything you need to know, from AI to iOS 18 and beyond. All the talking points you won’t want to miss, easily digestible in five minutes so you can skip the two-hour keynote speech. Here goes.
Apple Intelligence
AI is one of the hot topics of 2024, and Apple is one of the last big tech companies to take a swing at it. Or so it seemed.
A privacy-first AI
In a bid to change that narrative, Tim Cook highlighted Apple’s long-standing use of AI under the hood before introducing its next evolution, smugly dubbed Apple Intelligence. This new suite of AI-powered features aims to make Apple devices more powerful, intuitive, integrated, and personal, all while maintaining Apple’s commitment to privacy.
Apple Intelligence is designed to understand you and your personal context, providing more helpful and relevant responses. The system is built with privacy from the ground up, leveraging on-device processing made possible by Apple Silicon chips. This ensures that your data remains secure and private. For tasks requiring more power, Apple will provide secure cloud-based AI servers that won’t store your data, setting a new standard in AI privacy.
That all sounds great from a security point of view, but what can Apple Intelligence actually do for the average user? Quite a lot, it seems.
Siri 2.0
Most notably, Siri has been given a huge boost. Its speech is much more natural, it understands relevant context and personal details better, and it can even take actions inside apps for you. Apple says it’s finally getting close to its original vision for what Siri should be. Thanks to a partnership with OpenAI, Siri can optionally defer to ChatGPT for tasks that are better suited to a more generalized AI, like general knowledge and content creation.
Writing tools
Speaking of content creation, new writing tools now permeate iOS, giving instant access to an AI model that can rewrite, proofread, and summarize text blocks. Anywhere you can write text, these tools can be used, and they go even further in the Mail app for summarizing and replying to emails. You’ll also get transcriptions and summaries in the Phone and Notes apps.
Generative imagery
Apple isn’t skipping the image-generation side of AI, either, despite it being a controversial subject amongst artists and creatives. Genmoji allows users to make an infinite variety of custom emoji from text prompts. A new Image Playground app allows users to create custom AI art in a variety of styles. In the Notes app, you can circle a rough sketch with the new Image Wand tool to turn it into a polished, finished image. All processing is done on-device, and it looks as though Apple paid a lot of money to license imagery from Shutterstock for the training data. It’s still pretty creepy though.
Intelligent Photos
Finally, there’s some neat stuff coming to the Photos app thanks to Apple Intelligence. The new Clear Up tool can remove background objects with a tap, while the search bar now understands natural language and you can even create custom Memories from a text prompt.
We’ll dig into every single AI-powered feature in more detail once we’ve had some time to play around with the developer betas. For now, let’s move on. Apple announced a ton of other stuff besides AI today.
iOS 18
Still with me? Here are the most important things Apple announced for iPhone users.
Customization
As rumored, users can now position app icons anywhere on the Home Screen. iOS 18 also brings new dark mode-specific icons that activate automatically, and users can select a tint color that affects every single app.
Control Center has been overhauled to offer more customization, with multiple themed pages accessible by swiping, and a controls gallery for organizing features, and there are more buttons available for the Lock Screen.
Privacy
There’s a big emphasis on privacy, too, with the ability to lock individual apps behind Face ID, Touch ID, or a passcode. Locked apps do not share their information, and you can even stash apps into a Hidden folder.
There are also more nuanced privacy options when connecting Bluetooth accessories or sharing contacts with apps.
Communication
There are some big changes in the Messages app, not least the addition of RCS support to improve communication with your green-bubbled Android friends. Other changes include an expanded range of tapbacks, text formatting, and new text animation effects.
You can now schedule texts to send at a later time, and the technology behind Emergency SOS has expanded to Messages via Satellite, allowing you to send encrypted texts even when off-grid and without a signal.
The Mail app now includes automatic inbox sorting, categorizing emails into primary and secondary inboxes based on importance.
Safari
Safari’s new Highlights feature uses machine learning to detect and emphasize relevant information as you browse, offering summaries, quick links, contact info, and even the ability to listen to music from someone you’re reading about.
An improved Reader View now includes a table of contents and an AI-generated summary of the entire page, making content easier to navigate and digest.
Photos
With iOS 18, Photos app undergoes its “biggest redesign ever,” offering a unified layout that exists in a single view. At the top, you’ll find a photo grid, adjustable via new tabs and filters. Below, the app provides quick access to auto-generated collections such as memories, trips, recent days, and people.
The app’s customizable nature allows you to reorder collections and surface the items you care about most to the top of the list.
More
There were also plenty of smaller changes that flashed by during the Keynote: detailed hiking routes in Maps; Tap to Cash for quick and private Apple Cash transactions; redesigned event tickets in Wallet; a stat-heavy new Insights view for the Journal app; a high-performance Game Mode for better gaming; guest access and robot vac support in Home; integration between the Calendar and Reminders apps.
iPadOS 18
As ever, most of what’s new for iPad is the same stuff that’s new for iPhone. But there were two iPadOS 18 exclusive features that really stood out…
Math Notes
After 15 years AWOL, the Calculator app finally arrives on iPad. Initially resembling the iOS version, the app reveals a new feature called Math Notes when used with the Apple Pencil. Users can write out numerical expressions and sums like on paper, instantly seeing the answers in handwriting that matches their own. You can use variables, save scratch pads of calculations, and generate graphs.
Smart Script
Math Notes are available in the Notes app, too, along with an awesome new Apple Pencil feature called Smart Script. On-device machine learning recreates your handwriting style, allowing you to scribble at speed while maintaining a neat, organic appearance. Handwriting is now as flexible as typed text, enabling copy-pasting, spellchecking, and erasing.
More
A few more iPadOS tidbits: a new floating tab bar that morphs into the sidebar when more buttons are needed; SharePlay users can tap and draw on their contact’s screen and even remotely control another person’s device; and Freeform adds Scenes.
macOS 15
This year’s macOS update is named macOS Sequoia, bringing a host of new features and improvements alongside many of the same iOS and iPadOS changes we discussed earlier.
iPhone Mirroring
Another Continuity feature, iPhone Mirroring allows you to see a live, fully interactive replica of your iPhone on your Mac. This integration includes iPhone notifications on the Mac, and seamless drag and drop between the devices. The screen on your actual iPhone remains locked the whole time, ensuring privacy.
Passwords
Say goodbye to Keychain and hello to the new Passwords app. This app unifies all your login credentials, security alerts, verification codes, and autofill data into one convenient place. It syncs automatically across macOS, iOS, and iPadOS, ensuring you always have access to your important information.
More
There’s plenty more, with our favorites including Zoom-like virtual backgrounds in FaceTime, and auto-layout when you ‘snap’ apps to the edge of the screen.
watchOS 11
The latest update for Apple Watch brings a suite of improvements.
Activity Tracking
watchOS 11 introduces a personalized ‘effort rating’ to help you plan your workouts more effectively. This new metric, personalized by your own workout data, allows you to monitor your training load, giving you insights on whether to push harder or take it easy.
You can also adjust your activity ring goals based on the day of the week and pause for a few days without losing your streak.
Health
A new Vitals app helps you keep track of your most important health metrics and alerts you if anything appears abnormal.
The Health app also includes new views for pregnancy tracking, as well as baby sleep tracking, providing comprehensive support for expectant and new parents.
More
That’s not all. watchOS 11 also features: a more intelligent smart Stack widget; support for dictating to the Translate app; integration with safety feature Check In; and watch face customizations curated from your best photos.
visionOS 2
Following the launch of Vision Pro in February, Apple is already rolling out visionOS 2, highlighting the rapid evolution of the platform. It seems Apple wants to react quickly to any teething problems and keep momentum.
New features
We’ll keep this one brief, as I estimate less than 0.1% of you actually own a Vision Pro headset. The latest software makes it possible to generate faux-3D Spatial Photos from a normal photo, and there’s new Immersive Video content on the way. You can now make a huge super-widescreen virtual display with which to control a paired Mac.
New hand gestures have been introduced for quick access to the Home Screen and Notifications, and you can finally rearrange app icons. Meanwhile, improved dev tools aim to streamline to development process, which should help expand Vision Pro’s relatively slim App Store catalog.
Availability
Currently, Vision Pro is only available in the US, but that’s about to change. The product is coming to 8 new regions over the summer: China mainland, Hong Kong, Japan, and Singapore will see availability from June 28, with Australia, Canada, France, Germany, and the UK following suit on July 12.
tvOS 18
Apple TV doesn’t get many new features these days. Nevertheless, the latest iteration introduces InSight, a feature that provides real-time information on actors, characters, and music on screen when you swipe down on the remote. The enhanced dialog feature now supports a wider range of speaker setups, ensuring clear speech even during action scenes.
Additionally, tvOS now supports the filmic 21:9 widescreen ratio and offers more choices for TV screensavers, including portrait photos, clips from TV+ shows, and a charming Snoopy and Woodstock animation. Perhaps more impressive is the fact that this latest update remains compatible with the 9-year-old Apple TV HD.
AirPods
Using Siri with AirPods is now easier, as you can respond with gentle nods or shakes of your head. Perfect for silently declining calls in a busy room.
Meanwhile, Voice Isolation comes to AirPods Pro, removing background noise around you for clearer audio. Personalized spatial audio is now available for gaming sessions, providing a more immersive experience.
Availability
All of the new software updates, including macOS Sequoia, iOS 18, iPadOS 18, watchOS 11, and more, will be available to all users this fall.
However, for those eager to get a head start, a public beta will be available starting in July, and the developer beta is available immediately. This means we can start testing these exciting new features right now and share more insights as we explore them. Stay tuned!