Copilot+ PCs are the first PCs to run Small Language Models (SLM) on-device. It means these AI models are powered by local processing capabilities rather than relying on cloud, which is typically done by AIs like ChatGPT. This approach can result in faster responses as compared to the cloud-based Copilot or ChatGPT.
Now, Microsoft has introduced AI Dev Gallery, which makes it easy to add on-device AI features to any app.
The AI Dev Gallery app is for developers who want to try multiple models to integrate an AI feature into their app. AI Dev Gallery has over 25 samples to download and run on Windows 10. Moreover, you can export the project or source code directly into your app and get it working instantly.
It works with Windows 10 and 11 and supports x64 and ARM64 architectures.
Out of curiosity, Windows Latest cloned the AI Dev Gallery from its GitHub repository and compiled our own version of AI Dev Gallery.
Currently, the only way to access it is by building the project in Visual Studio and then running it. In addition, you’ll need at least 20GB of space and a multi-core CPU. It also suggests a GPU with 8GB VRAM, but those are for heavy models only.
We started with a 4-core CPU, 4GB RAM, or a Windows 11 PC. The app has two modes: Sample and models. We used the sample view to check out all the available models.
The app categorizes them into Text, Image, Code, Audio and video, and Smart Controls.
Testing the models
Models for image generation and video generation need about 5GB of bandwidth to download and save it to your PC. To better understand how things work, we started with a small one related to image upscaling, which was under 100 MB.
We took a screenshot of the screen and tried to upscale it using the CPU. Note that you can switch between CPU and GPU for processing the request.
The upscaling happened in under 30 seconds in this inferior virtual machine, where the RAM usage increased to 1GB. As you can see in the screenshots below, App Gallery produced an upscaled version of the image with a 9272*4900 resolution.
However, the 100MB upscale model has its own issues. While it used our local resources to upscale images, elements, especially text, were ruined. You can barely read the texts.
There’s no option to preview this generated image in a bigger window or full-screen. Even the download option isn’t there to save it to disk.
We tried another model called Detect Human Pose. It identifies someone’s position in an image. While it accurately identified a basic walking person, it even started showing position marks for screenshots of our desktop with a few apps open.
We don’t know how these models can be integrated into the app, but some of these features can be run locally. Of course, PCs will need bigger storage for these models and powerful CPUs with 16GB or more RAM.
What do you think? It is worth downloading a 5GB model to convert a text prompt into an image or wait for 30 seconds to do that on a web app. Clearly, most of these features have very niche use cases and implementation environments rather than appealing to the whole Windows 11 user base.
The post We tried Microsoft’s new app to run AI models on Windows 11 locally. It’s really good appeared first on Windows Latest
Source: Read MoreÂ
