Apple Intelligence Foundation Language Models
Apple has developed language models to enhance its Apple Intelligence features, including a compact on-device model and a larger server-based model, emphasizing Responsible AI and improving user interactions in iOS and macOS.
Read original article
Apple has developed foundation language models to enhance its Apple Intelligence features, which were introduced at the 2024 Worldwide Developers Conference. These models include a compact, approximately 3 billion parameter model optimized for on-device use and a larger server-based model intended for Private Cloud Compute. The report details the architecture, training data, training process, optimization for inference, and evaluation results of these models. Emphasizing Responsible AI, Apple integrates ethical principles throughout the development process. The Apple Intelligence system, deeply embedded in iOS 18, iPadOS 18, and macOS Sequoia, comprises multiple generative models tailored for everyday user tasks. These models can adapt in real-time to assist with activities such as text writing and refinement, notification summarization, image creation for social interactions, and facilitating in-app actions to streamline user experiences. Additionally, Apple hosted a Natural Language Understanding workshop earlier in 2023, which gathered experts from academia and industry to discuss advancements in the field. The workshop featured talks and discussions that highlighted the current state of natural language understanding. Overall, Apple's ongoing research in machine learning aims to push boundaries and improve user interactions through advanced AI technologies.
Related
Apple Wasn't Interested in AI Partnership with Meta Due to Privacy Concerns
Apple declined an AI partnership with Meta due to privacy concerns, opting for OpenAI's ChatGPT integration into iOS. Apple emphasizes user choice and privacy in AI partnerships, exploring collaborations with Google and Anthropic for diverse AI models.
Apple just launched a public demo of its '4M' AI model
Apple publicly launches its '4M' AI model with EPFL on Hugging Face Spaces, showcasing versatile capabilities across modalities. The move signals a shift towards transparency, aligning with market growth and emphasizing user privacy amid ethical concerns.
Apple Intelligence's Privacy Stacks Up Against Android's 'Hybrid AI'
Apple introduced "Apple Intelligence," an AI system with OpenAI, sparking privacy debates. Apple's Private Cloud Compute prioritizes privacy, contrasting Android's hybrid AI approach. Experts praise Apple's privacy features, but concerns persist over user data security.
Apple Intelligence Now Available in New iOS 18.1, iPadOS 18.1, Sequoia Dev Betas
Apple has released developer betas for iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1, featuring Apple Intelligence tools. Compatibility requires specific devices, with additional features expected later this fall.
Apple releases first preview of its long-awaited iPhone AI
Apple has launched Apple Intelligence, enhancing Siri and automating tasks in iOS 18.1 beta. Key features include a redesigned interface and AI-generated summaries, with public release expected later this year.
7 comments1. Two Main Models: - AFM-on-device: ~3 billion parameters, for efficient on-device use - AFM-server: Larger model for Private Cloud Compute
2. Architecture and Training: - Based on Transformer with optimizations - Three-stage training: core, continued, and context-lengthening - LoRA adapters for task-specific fine-tuning - Innovative quantization: 3.5-3.7 bits per weight
3. Performance and Benchmarks: - AFM-on-device outperforms larger models (e.g., Gemma-7B, Mistral-7B) - AFM-server competitive with GPT-3.5 - HELM MMLU (5-shot): AFM-on-device 61.4%, AFM-server 75.4% - GSM8K (8-shot CoT): AFM-server 83.3% - Strong in instruction-following (IFEval) - Best overall on Berkeley Function Calling Leaderboard
4. Capabilities: - Excels in instruction following, tool use, writing, math - Long context support up to 32k tokens - Specialized for tasks like summarization
5. Responsible AI: - Focus on user privacy and responsible AI principles - Extensive safety measures (red teaming, human evaluations) - Lower violation rates on safety prompts vs. other models
6. Unique Aspects: - "Accuracy-recovery adapters" post-quantization - Novel RLHF framework: "Iterative Teaching Committee" (iTeC) - New RL algorithm: MDLOO
Related
Apple Wasn't Interested in AI Partnership with Meta Due to Privacy Concerns
Apple declined an AI partnership with Meta due to privacy concerns, opting for OpenAI's ChatGPT integration into iOS. Apple emphasizes user choice and privacy in AI partnerships, exploring collaborations with Google and Anthropic for diverse AI models.
Apple just launched a public demo of its '4M' AI model
Apple publicly launches its '4M' AI model with EPFL on Hugging Face Spaces, showcasing versatile capabilities across modalities. The move signals a shift towards transparency, aligning with market growth and emphasizing user privacy amid ethical concerns.
Apple Intelligence's Privacy Stacks Up Against Android's 'Hybrid AI'
Apple introduced "Apple Intelligence," an AI system with OpenAI, sparking privacy debates. Apple's Private Cloud Compute prioritizes privacy, contrasting Android's hybrid AI approach. Experts praise Apple's privacy features, but concerns persist over user data security.
Apple Intelligence Now Available in New iOS 18.1, iPadOS 18.1, Sequoia Dev Betas
Apple has released developer betas for iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1, featuring Apple Intelligence tools. Compatibility requires specific devices, with additional features expected later this fall.
Apple releases first preview of its long-awaited iPhone AI
Apple has launched Apple Intelligence, enhancing Siri and automating tasks in iOS 18.1 beta. Key features include a redesigned interface and AI-generated summaries, with public release expected later this year.