Top News
A Bunch of New Open Source LLMs!
Nvidia’s Llama-3.1-Minitron 4B is a small language model that punches above its weight
Nvidia’s research team has developed a small language model (SLM), Llama-3.1-Minitron 4B, that performs comparably to larger models while being more efficient to train and deploy. The team used techniques of pruning and distillation to create this model. The team started with the Llama 3.1 8B model, fine-tuned it on a 94-billion-token dataset, applied depth-only and width-only pruning, and then fine-tuned the pruned models using NeMo-Aligner. The resulting Llama-3.1-Minitron 4B model performed close to other SLMs, despite being trained on a fraction of the training data. The width-pruned version of the model has been released on Hugging Face under the Nvidia Open Model License for commercial use.
Microsoft reveals Phi-3.5 — this new small AI model outperforms Gemini and GPT-4o
Microsoft has unveiled Phi-3.5, the latest version of its small language model, which outperforms other small models from Google, OpenAI, Mistral, and Meta on several key metrics. Phi-3.5 is available in 3.8 billion, 4.15 billion, and 41.9 billion parameter versions, all of which can be downloaded for free and run using a local tool like Ollama. The model excels in reasoning and math benchmarks, surpassing competitors like Llama and Gemini. Phi-3.5 also comes in a vision model version that can understand images, and a mixture of expert models that split learning tasks across different sub-networks for more efficient processing.
AI21 Introduces the Jamba Model Family: The most powerful and efficient long-context models for the enterprise
AI21 has announced the release of two new open models, Jamba 1.5 Mini and Jamba 1.5 Large. These models, built on a hybrid architecture that combines the strengths of Transformer and Mamba architectures, offer unmatched speed, efficiency, and performance in long-context language models. The Jamba 1.5 Large, a Mixture-of-Experts (MoE) model with 398B total parameters and 94B active parameters, is designed to handle complex reasoning tasks with high quality and efficiency. Both models utilize a true context window of 256K tokens, the largest currently available under an open license, and have demonstrated superior performance in latency tests against similar models.
Open source Dracarys models ignite generative AI fired coding
Abacus.ai, an AI model development platform, has introduced a new family of open large language models (LLMs) for coding, named Dracarys. Unlike its previous general-purpose LLM, Smaug-72B, Dracarys is specifically designed to optimize coding tasks. The “Dracarys recipe” has been applied to the 70B parameter class of models, involving optimized fine-tuning techniques to improve the coding abilities of any open-source LLM. According to LiveBench benchmarks, the Dracarys tuned version significantly improves the performance of existing models.
Major Developments with California’s SB 1047 AI Regulation Bill
Anthropic CEO Backs California AI Bill, But Still Has Concerns
Anthropic has expressed support for the amended version of California’s AI bill SB 1047, despite some reservations. CEO Dario Amodei believes the benefits of the bill, which requires AI companies to adopt and disclose safety and security protocols, outweigh its costs. However, he also notes that the bill could disproportionately affect larger AI companies and potentially lead to overreach by the Attorney General. The bill is set for a final vote in the Assembly by August 31. If passed, it will be sent to Governor Gavin Newsom for approval or veto by September 30.
Ex-OpenAI researchers claim Sam Altman’s public support for AI regulation is a facade: “When actual regulation is on the table, he opposes it”
OpenAI has opposed a proposed AI bill (SB 1047) aimed at implementing safety measures to prevent AI technology from causing potential harm. This opposition has sparked criticism from former OpenAI researchers William Saunders and Daniel Kokotajlo, who argue that the development of advanced AI models without proper regulation could lead to catastrophic consequences. OpenAI CEO Sam Altman has publicly advocated for AI regulation, but the researchers suggest that his support is superficial, as he opposes actual regulatory measures when they are proposed. OpenAI’s Chief Strategy Officer, Jason Kwon, has argued that AI regulation should be implemented at the federal level to foster innovation and establish global standards, but it remains uncertain whether the bill will be passed or if OpenAI’s proposed amendments will be incorporated.
OpenAI’s opposition to California’s AI bill ‘makes no sense,’ says state senator
Elon Musk unexpectedly offers support for California’s AI bill
Ars Technica content is now available in OpenAI services
OpenAI has announced a partnership with Condé Nast, the parent company of Ars Technica, to display content from its publications within AI products such as ChatGPT and SearchGPT. This partnership will allow users of these AI services to access information from Condé Nast publications, and will also enable OpenAI to use this content to train future AI language models. The training process, which is computationally intense and expensive, involves feeding content into an AI model’s neural network to improve its ability to process conceptual relationships. Despite this partnership, Condé Nast’s internal policy still prohibits the use of text generated by AI in its publications. The deal is seen as a strategic move by Condé Nast to expand its content reach, adapt to changing audience behaviors, and ensure proper compensation and attribution for its intellectual property.
Other News
Tools
Google Releases Powerful AI Image Generator You Can Use for Free – Google has released a powerful AI image generator, Imagen 3, for free use in the U.S., which outperforms other models and offers advanced editing options, but raises concerns about the data used for training and copyright issues.
OpenAI Launches Fine-Tuning Feature for GPT-4o – OpenAI has launched a fine-tuning feature for GPT-4o, allowing developers to increase performance and accuracy for their applications by customizing the model’s responses with their own datasets.
Perplexity’s latest update improves code interpreter, charts included – Perplexity’s latest update improves its code interpreter, allowing for the installation of libraries and display of charts in the results, expanding its use cases.
Perplexity adds Flux.1 model for Pro users alongside Playground v3 update – Perplexity introduces new image generation models for Pro users, including the Flux version 1 model, alongside an update to Playground v3.
Ideogram AI expands its features with v2 model and color palette options – Ideogram AI has released its new text-to-image model v2.0, offering improved text rendering, color palette options, and five different models to cater to a wider range of AI-produced content.
Lightricks’ LTX Studio, the AI Visual Storytelling Platform, Now Open to All – Lightricks has announced the public availability of LTX Studio, an AI-driven storyboarding and prototyping platform designed for creative film and marketing professionals, offering real-time generative and editing solutions, enhanced collaboration, character acting, and generation control.
McAfee introduces AI deepfake detection software for PCs – McAfee introduces AI deepfake detection software for PCs, utilizing advanced AI models trained on nearly 200,000 video samples to quickly and privately determine whether a video has been manipulated, all while maintaining user privacy and device performance.
Zed Editor Adds Anthropic-Powered AI Features – Zed, the text editor taking the development world by storm, has announced new AI features powered by Anthropic’s Claude. Zed is a new text editor written entirely in Rust, benefiting from the speed, security, and other features the language provides.
Business
Waymo launches its latest driverless ride-hailing platform – Waymo launches its 6th-generation driverless ride-hailing vehicles, featuring reduced production costs, increased range, compute power, and fewer sensors.
Waymo is now giving over 100,000 paid self-driving rides per week – Waymo has surpassed a new milestone with its paid driverless ride-sharing, celebrating over 100,000 paid self-driving rides per week and unveiling its next-generation ride-hailing vehicle platform.
Waymo wants to chauffeur your kids – Waymo is considering a subscription program called “Waymo Teen” that would allow teenagers to hail its cars solo and send pickup and drop-off alerts to their parents.
Creatopy, which automates ad creation using AI, raises a $10M Series A – Creatopy, a startup that automates ad creation using AI, has raised a $10 million Series A and now serves over 5,000 brands and agencies, focusing on scaling, personalization, and automation at scale for digital ads.
Cruise’s robotaxis are coming to the Uber app in 2025 – Cruise, General Motors’ self-driving subsidiary, has signed a multi-year partnership with Uber to bring its robotaxis to the ride-hailing platform in 2025, following a safety incident and regulatory challenges.
Salesforce Doubles Down On Autonomous Agents With Einstein SDR and Sales Coach – Salesforce introduces Einstein SDR and Sales Coach agents, which autonomously engage with prospects, nurture the pipeline, provide personalized coaching, and aim to help sales teams accelerate growth.
Tesla is hiring workers for $48 an hour to wear motion-capture suits to train its humanoid robots – Tesla is hiring workers to wear motion-capture suits and gather movement information to train its humanoid robots, which are designed to automate work in company factories.
Perplexity AI to launch ads on search platform by fourth quarter – Perplexity AI plans to introduce advertising on its AI-powered search platform by the fourth quarter, following a successful fundraising round and partnerships with major publishers.
China’s ambitions in humanoid robots in full display at expo – China showcases its progress and ambitions in humanoid robots at the World Robot Conference, with companies unveiling advanced robots powered by large language models and aiming to dominate the field.
Chinese firms bypass US export restrictions on AI chips using AWS cloud – Chinese firms are using cloud services from American companies to bypass US export restrictions on AI chips, allowing them to access restricted technologies.
Research
Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation – A scalable and flexible transformer-based policy called CrossFormer is proposed to train a single policy across various robot embodiments, demonstrating its ability to control vastly different robots and outperform prior state-of-the-art methods in cross-embodiment learning.
Transformers in music recommendation – AI-driven music recommendation systems need to consider the broader context of a user’s preferences and activities to provide more accurate and valuable song recommendations.
Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model – A new multi-modal model called Transfusion combines language modeling and diffusion to train a single transformer over mixed-modality sequences, scaling significantly better than quantizing images and training a language model over discrete image tokens.
LongVILA: Scaling Long-Context Visual Language Models for Long Videos – Scaling Long-Context Visual Language Models for Long Videos through the introduction of LongVILA, a full-stack solution for long-context vision-language models, including system, model training, and dataset development.
To Code, or Not To Code? Exploring Impact of Code in Pre-training – Including code in pre-training data significantly improves general language model performance across a wide range of tasks, beyond just coding-related ones.
Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models – Transformers can be distilled into subquadratic state space models (SSMs) using a method called MOHAWK, allowing SSMs to benefit from the computational resources invested in training Transformer-based architectures.
Meta AI Proposes ‘Imagine yourself’: A State-of-the-Art Model for Personalized Image Generation without Subject-Specific Fine-Tuning – Personalized image generation is gaining traction due to its potential in various applications, from social media to virtual reality. However, traditional methods often require extensive tuning for each user, limiting efficiency and scalability.
TableBench: A Comprehensive and Complex Benchmark for Table Question Answering – Advancements in Large Language Models have improved the processing of tabular data, leading to the creation of a comprehensive benchmark called TableBench to address the challenges of applying LLMs in industrial scenarios.
Training-free Graph Neural Networks and the Power of Labels as Features – Training-free Graph Neural Networks can leverage the power of labels as features, eliminating the need for extensive training.
Concerns
Google DeepMind staff call for end to military contracts – Google DeepMind employees call for an end to military contracts due to concerns about the use of AI technology for warfare, urging the company to investigate and cut off military access to their technology.
Google’s AI ‘Reimagine’ tool helped us add wrecks, disasters, and corpses to our photos – Google’s new AI photo editing tool, Reimagine, allows users to add realistic and disturbing elements to their photos, raising concerns about the potential for misuse and the lack of safeguards to identify manipulated content.
Authors sue Claude AI chatbot creator Anthropic for copyright infringement – Authors are suing AI startup Anthropic for training its chatbot Claude on pirated copies of copyrighted books, alleging “large-scale theft” and challenging the company’s claim of responsible and safety-focused AI development.
Policy
Fake Biden Robocalls Cost Wireless Provider $1 Million in FCC Penalties – Fake Biden robocalls cost wireless provider $1 million in FCC penalties after allowing deepfake calls to be transmitted during New Hampshire primaries, leading to a settlement and new network security measures.
Analysis
We finally have a definition for open-source AI – Open-source AI is defined as a system that can be used, inspected, modified, and shared without restrictions, addressing the lack of a standard definition and potential misuse of the term by companies.
Source: Read MoreÂ
