Hands-On Windows 128 transcript

Please be advised this transcript is AI-generated and may not be word for word. Time codes refer to the approximate times in the ad-supported version of the show

0:00:00 - Paul Thurrott
Coming up. Next, on Hands-On Windows, we're going to look at one of the most controversial features ever introduced in Windows and another fun AI tool that I think you're going to like a lot Podcasts you love From people you trust. This is Twit. Hello everybody and welcome back to Hands-on Windows. I'm Paul Thrott and this week we're going to look at two of the premier new AI tools coming to Windows 11.

Both of these features require a co-pilot plus PC and If you want to use them now as I record this in early 2025, you actually have to enroll your PC in the Insider Program. It's not for everybody, but sometime this year these features will make their way into Windows, into stable, so to speak. They will require a copilot plus class PC, but they'll be more broadly available. So just a quick recap this computer I'm using here today is a Surface laptop, so it's a co-pilot plus PC running Snapdragon X Elite. So this is the ARM-based version of Windows, but it looks and works, of course, like Windows does. It has that 16 gigabytes or more of RAM, I think it's 512 gigabytes or more of SSD storage, a 40 plus TOPS MPU. So it meets that co-pilot plus PC specs and if you think back to I don't know mid-2024 and beyond. There was this set of features that these types of computers could get right, so the full Windows Studio effects for mostly video, but also a little bit of audio, live captions with real-time translations. Co-creator capabilities in paint, restyle images and some background removal stuff as well in photos. Auto SR super resolution in games Not that that helps too much.

On Snapdragon, and then there was Recall. Remember Recall? So Recall was announced in May last year and then delayed, and then I think it was November 2024, they finally put it out in preview on Snapdragon X-based co-pilot plus PCs. So a couple of big changes since then. One, those co-pilot plus PC experiences are starting to come to Intel and AMD-based Copilot plus PCs right, so you'll start seeing that. And the Copilot I'm sorry, the Recall preview is now available on those computers as well. But Microsoft also announced a new set of features that will be unique to CoPilot plus PC, and over the next couple of videos and then beyond, I'm going to look at some of those. So Recall we knew about, but there's a related feature called Click to Do, which I think is actually a lot more interesting and is going to benefit a lot more people than Recall, and so we're going to look at those two today in this video and then in future videos we'll look at some of those other new features.

So recall the way that this works is you sign into Windows for the first time, you'll get a promotion, basically for it you can opt in. If you don't opt in, it's not running in the background anymore. That was the original plan. I'll just bring up the UI so you can kind of see it. It does this Windows Hello thing. I have to look over here. That's where my webcam is, and if you saw that, you would have seen also what is a new Windows Hello experience. Right, that's new to 24H2. So here we can see the other screens.

These are my notes and the recording that we're doing here, but I can go back in time and click around. So we're going to look at this. I'll just try to find a safe place to put this. There's my notes from today actually. So we've got I think there's probably.

Let's actually see there's some amount of data in here. So I have some days, some weeks, whatever it is of data. Okay, so here's a document I was working on recently. Okay, we'll just leave it there. So this is just an example of a recall, what they call a snapshot.

But before we get into the UI and how we might use this and so forth, I just wanted to go over some of the kind of management-related stuff. So if you go into the Settings app and you go to Privacy and Security, you'll see this Recall and Snapshot section, which is new. Now, right, if you have this capability, you'll see this in settings. Like I said, by default this thing is off, but I've turned it on and when you turn it on it will allot a certain amount of storage based on how much storage your PC has and also the free storage. So I just downloaded an enormous hundreds of gigabytes Google takeout file and I. This used to be about 1.5 gigabytes. That was reserved because I'm taking up more space. Now on the disk it's gone down toa little bit under a gigabyte, right, or? I'm sorry, that's the amount I'm using. Excuse me, oh, it is 100. I'm sorry. This has not changed. It's 150, so this is the amount that's put aside. You can change it. This particularly disk has a one terabyte drive, so actually that hasn't changed. That's pretty good.

You can decide how long you want to keep these snapshots. You can go in and delete snapshots as a snapshot management capability, and then they have these filters. So there are automatic filters and then there are manual filters. So the automatic filters are based on heuristics and some rules. Microsoft will try to hide sensitive data, so things like social security numbers, credit card numbers, etc. In my experience so far this is a preview, I haven't actually experienced it doing that successfully, but it's supposed to and you can also add your own apps or websites to the filter so that they're not recorded. So you'll never get a screenshot of those items. If you want to keep that private and not that anyone could actually get into this. But if you're worried about whatever it is whatever it is you're doing, no one's asking questions you can hide that from recall.

So as recall runs, you can see this little icon down here in the tray and that just tells you it's saving snapshots. It has a little flyover here where you can open it. You can pause it for some amount of time, you can go in and access these settings which you can see here, etc. I've already opened recall, so I don't have to open it there. This is the basic UI.

If you're familiar with file history the feature, I think it debuted in Windows 7, and it's still in Windows today. It's not used as much. But Windows Timeline, another feature that has come and gone. That one debuted in Windows 10, gone in Windows 11, but kind of the same style timeline-based UI. You can go back and forth in time. You could go snapshot by snapshot.

You can see here I'm working on this document. I'm working on the document. You can see as things change on the screen. It's taking a screenshot. Taking a screenshot doesn't seem that sophisticated, but a big part of Recall is these local what they call small language models that are downloaded to your drive the first time that you install or enable this feature. It's actually a little tedious to install them. You have to do them one by one. It's very strange, but they go through Windows Update. So if I go into Windows Update history, I think they are under AI component updates, of course. So the day I recorded this I actually got an update to all four of these models. It used to be three models when this feature first arrived, it was three. Now there are four and you can see here the top four of those things. So image extraction, image search, semantic analysis and PhiSilica, which is a Microsoft LLM that's been brought down to a smaller size. So these are what's necessary to do the work that Recall does, and so what that work is is essentially image and text recognition. So the point of the screenshot is that as you're working on your computer, things change. It takes a picture. You're working on your computer, things change. It takes a picture. You're working on your computer, things change. It takes a picture. Now you've got some body of work and so, like I said, you can click through here and look for things like this.

I don't actually think this is how most people are gonna use this product or this app or service or whatever you wanna call this. But I think the real point here is search. Right that you know you did something on your computer but you can't quite remember it, right? So the week that I write this, I know that Sonos has been in the news a lot, for example. So I can type in Sonos into the search box and I can search. And when you get results in recall, the results are in two batches, right? So you get text matches and you also get visual matches, and when there are multiple apps, it will actually give you a chance to filter per app, and so here you can see, these are articles I wrote about Sonos this past week. They had revenues, they had a restructuring, they there was a rumor in the verge about them coming out with the first set top box, so this is a way to go back to that thing. I saved this somewhere I don't maybe remember where it is. This is a kind of a neat way to get back to it, right, and so there's some a number of Snapshots in there and then visual matches, are they? They're usually images, right, and this one's kind of interesting because this is actually a Sonos speaker. So it knew, looking at that speaker and analyzing that image, that that wasn't just a speaker, it was in fact a Sonos speaker, right. So that's pretty smart and that's what those SLMs are doing.

I'll do another. I've read about Microsoft a lot, as you might imagine, so I will do a search for Microsoft and it will be similar, I would imagine. Yeah, so I've got text-based. Well, that's interesting. This is text inside of an app, but mostly, hopefully, images of apps where Microsoft appears. In this case, it's a website, and there are more. They go off to the side. There's going to be a bunch of them, I bet, because I write about Microsoft all the time and then visual match is the same thing, like here's a Microsoft presentation from before Windows 10 was released, actually Some stuff from my website, et cetera. So it's analyzing the images and trying to pull things out. It's not because it says Microsoft somewhere there and actually I don't think it does, or maybe it does somewhere but it's doing that kind of matching capability where it's trying to find images based on specific criteria. So I will bring up let me see which one of these actually, none of these. Let me go back to my. Yeah, I'll bring up a Sonos.

Well, that's not a complete article, so that's okay. Here's something that's in place, right? So this is kind of interesting. This is a snapshot. This is a picture of a text editor, essentially where I was writing an article. It's got hyperlinks, et cetera. It has highlighted all the text text and this feature.

Here is something called click to do and when Microsoft first announced and then released this in preview, it only worked within recall, which is this app we're using here. So the way this works is it analyzes what's on screen and if it's a text, it will select it. In this way, it's got that kind of purple, pink, ai halo effect that we're getting kind of used to, right. If there was an image in here, it would do the same for the image. I've got this kind of fun cursor now, but I can click on any of these things to select, I can right-click and then I get certain actions. So in this case it's just copy and open with. It's not particularly interesting, but there are other actions you can do against text and those things become much more interesting when you're outside of recall, and we're going to get to that in a moment.

But if I copy this to the clipboard and then paste it, you can see it's actually pulled in the text. That's in the menu of the app as well, right, I'm doing a very kind of dumb copy. It's describing everything, right. I'm doing a very kind of dumb copy. It's describing everything right, but this is all of the text that's in there. So it's doing this image recognition, ocr capabilities, right, putting it in the clipboard as text and then allowing me to use it in some other app, in this case Notepad. So you know, useful, not revolutionary or anything but interesting. So that's basically recall.

Now, there was a bunch of controversy around this app for some reason. It's a little strange, right, but I think this is pretty tame. I mean, depending on how you work, you may or may not find this useful, right? So in this case, I searched for blue item just to get a color in here, and you can see that these images include items that are blue, like this blue keyboard, this blue background. This is a blue hard drive, right? I mean, it's kind of interesting. Depending on how you think and depending on your work, you might find this really useful. I'm really organized with my files and everything. I actually know where everything is, so I don't really need this kind of thing personally, but I do think it's something that will benefit a lot of people. That said, I think that click to do by itself is actually even more interesting and, like I said up front, I think this is something that's going to benefit more people in the long run.

So ClickToDo inside of Recall, you know it's okay, but ClickToDo gives you access to these AI capabilities anywhere in Windows, in any application, and it works with text and images, and I think that's far more powerful. So, to show you that, I will do a couple of things. First, I will bring up a text document. So I'm actually going to bring up a. Let me bring up one. I've already written a live document, so I was writing about GitHub earlier today. So again, text document text editor, et cetera, et cetera.

The way that this works is you hold down the Windows key and then you click and you can see I've got that. Now I've got that fun little AI icon. And you click somewhere it does the little animation. Now it's selecting everything. I probably actually let me do it, let me do this in a way that I am only selecting part of this. You can kind of escape out of this. Let me just select this so that might make a little more sense, just to kind of keep it to the. It's not going to do it, but but okay, so we have other options here. So remember before and recall, copy yes, open with right, whatever apps I might have on the system that are compatible with this format, this text, but then you have these AI text actions, right? So summarize.

I can summarize the document that I wrote, which is itself a summary, in a way of a news item. And then there are these rewriting tools, not as extensive yet as those tools we looked at in that preview version of Notepad a couple of episodes ago. That was another AI feature that's coming down the pike pretty soon for Windows 10, or Windows 11, rather 24H2. But you could rewrite in a kind of a formal way, casual way, refine, et cetera. So I don't want to do this to the whole thing. So what I'm going to do is let me just find a one that doesn't have a quote, so it's just straight up writing I'll start a new document, paste, right, I'll do that thing again, and now I can generate a summary. Right Now. This isn't a lot of text.

Generally speaking, you would want to do a summary on something longer, right, a big PDF or whatever it might be, but still interesting it's. Arguably, this summary is roughly the same size as the thing I wrote. Again, that's just because it's not a great example, but you could also rewrite it to be more casual, to be more formal, just refine it. In this case, I'll just refine it. So this is just going to rewrite it, and this is you know, as we had the conversation with my with Notepad a few episodes ago is sort of what a blogger does, right? You take something that has been announced over here and you rewrite it and then you post it on your own site, and so this new feature lets you specify a set of files, which is exactly what I wrote in the use natural language, blah, blah, blah.

Okay, so actually that first sentence hasn't changed at all, but you can see, you know, there's, there's, there's an idea here of taking the words that I wrote and and shifting them somewhat. So that's, that's pretty interesting. But I think these other options are, depending on what you're doing here, might be more interesting. So is there a way to make this technical topic slightly more approachable by making it more casual? My writing here is a little stiff. This new feature lets you do this. They say you can tell they kind of reposition it in that way. So this is a good example, I think, of just kind of a small way that AI is going to start, you know, just making these little things a little easier, kind of like spell checking or grammar checking. Right, it's kind of an interesting idea. Okay, so that's text. But what about graphics? So we will discard you and we will close this or minimize that.

We're going to be doing an episode soon about some new photo features, so I do have some photos here. But that's okay, we can just bring this up. We'll put this over here. We can see it. So this is a photo of me and my dad. Well, you can see 1968, forever ago. The Photos app has AI features built in. We're going to look at those in probably the next episode, but we could just use click to do right. This could be in any app. And so here the options change because now we're looking at something that is an image and so there are slightly different options Copy, obviously, save, open with different apps that are compatible, and then you have these app-based AI features. So we're going to look at that stuff later. But interesting, right.

Visual search with Bing is actually pretty interesting too. Let me this is open to the wrong place. Let me pull that over up here, sorry. So this is open in Microsoft Edge. So this is kind of fun to me because this is a picture of a small child and a guy and skipping stones in a pond or whatever, and this is Bing Visual Search. So it's search the web for images that look like this image and you can see most of these are pretty close Kid guy, kid guy, you know. Kid guy, kid water, you know, with the water and the ripples, and you can kind of go down the list, right. That's, I mean, that's kind of fascinating. Frankly, it's pretty good.

I'm not sure what I would use that for personally, but if you have an image of some type and you're trying, you know, I want to find an image like this maybe you want to use it in some kind of a design or you're publishing to the web, like I do. I mean, this is rather, this is rather incredible, frankly, right. So it's kind of amazing. In fact, some of these are so close I'm wondering if those aren't pictures of my dad. It's like what's going on here? So that's pretty cool, right. So that's neat.

So that, to me, is going to evolve into something that I think is going to be much more compelling for more people because it works with everything. So this text editor that I'm using, for example, has basic spell checking. I don't think it has grammar checking per se. There are AI tools you can download to provide those right on top of the app, but the AI tools that are built into Windows, in particular, click to Do in this case work with everything, right. So I don't this document certainly doesn't need it but the ability to go in here and rewrite it if you're not a writer, if you're not really sure that this is what you mean to write, I think is really compelling. And the fact that it's just so universally available across applications, right, is amazing. Right, you do need a co-pilot plus PC, like I said up front. So there is that little wrinkle, but more and more PCs sold over the next year or so will be co-pilot plus PCs and then eventually that will just be all PC. So we're on the leading edge of this right now, but it's not going to be too long before this stuff is just available everywhere.

There are more features coming. This particular computer doesn't have them all. I'm enrolled in the Insider Program but they roll out features over time etc. So this semantic search which is going to be tied to Windows Key plus S is not available here on my computer but it will be soon. The Windows Key plus Q used to work for search. That's being remapped to click to do right. So now that, because it's a search of its own, in this case we're finding something on the screen. So that's interesting to me. I was very curious why they had two keyboard shortcuts for the same thing. Windows Key plus J, I think it is. Yeah, it came up on the other computer. I have to do the whole Windows Hello thing, but that's what launches Recall. So there you go.

Depending on the type of PC you have, you will either never get these features or you get them in the near future. My expectation is that Copilot plus PCs will see these features and stay able sometime in the first half of 2025, so it won't be too much longer. And if you don't have one, I'm not sure if this is enough to convince you to get one, but it's certainly something to think about. So I hope you found this interesting. We will have a new episode of Hands on Windows every Thursday. You can find out more at twittv slash how. Thank you so much for watching. Thank you especially to our Club Twit members. If you are watching this on YouTube and stuck with all the advertisements, please do consider subscribing to Club Twit. It's inexpensive and it's a ton of great content, not just this podcast, but many, many others.