Hugging Face's logo
Ujjwal Tyagi's picture
Building on HF
24 911 256

Ujjwal Tyagi

Ujjwal-Tyagi
mehmetalisezgin's profile picture Fishtiks's profile picture MachiaveIIi's profile picture
·

AI & ML interests

Chief Scientist at Shirova AI, focused on advancing open-source AI, Experienced in LLM fine-tuning, model architecture, and research, with a strong interest in building scalable and efficient models

Recent Activity

liked a dataset about 3 hours ago
nvidia/Nemotron-SFT-Safety-v1
repliedto reaperdoesntknow's post about 4 hours ago
# Three Teachers, One Student: Dual-Cognition Reasoning at 1.7B We distilled Qwen3-30B-A3B into 1.7B students that critique their own reasoning. H100, BF16, Apache 2.0. Here's our pipeline. **Stage 1 — Three Teachers, Three Profiles.** Same 30B base, three variants: Instruct (structured output), Thinking (extended deliberation), Coder (STEM decomposition). Each distillation uses proof-weighted KD — 2.25× amplified loss on reasoning tokens, decaying to 1.1×. The student learns *where to think harder*, not just what to output. **Stage 2 — Topology-Aware KD (TKD).** Standard KD treats the teacher's distribution as smooth. Language isn't smooth — it has topic shifts, reasoning pivots, register changes. We use Discrepancy Calculus to detect these structural boundaries, then amplify loss at jumps (3σ threshold) and cut training windows at low-discrepancy positions. The student preserves the teacher's structural knowledge, not just surface statistics. **Stage 3 — Ghost Imprinting.** Sequential distillation from different teachers leaves residual fields in weight space that neither teacher put there individually. The Cantor component of BV decomposition, applied to parameters. Models distilled Thinking→Coder exhibit deliberation patterns from the Thinking teacher that survived Coder overwriting. Emergent capability from structural residuals. **Stage 4 — DualMind.** One model, two voices, shared weights: ``` <explore> — free derivation, speculation <examine> — adversarial self-critique <response> — clean synthesis ``` The multi-model collision array collapsed into a single architecture. Role tokens, no extra parameters. For the full method: https://huggingface.co/reaperdoesntknow/DualMind_Methodolgy doi:10.57967/hf/8184.
upvoted a paper about 13 hours ago
SocialOmni: Benchmarking Audio-Visual Social Interactivity in Omni Models
View all activity

Organizations

AI FILMS's profile picture GEM benchmark's profile picture MusicAI's profile picture Open-Source AI Meetup's profile picture Chinese-Vicuna's profile picture East China Normal University's profile picture Keras Dreambooth Event's profile picture Stable Diffusion Dreambooth Concepts Library's profile picture Binghamton University's profile picture Blog-explorers's profile picture huggingPartyParis's profile picture LocalLLaMA's profile picture MLX Community's profile picture ONNX Community's profile picture Hugging Face Discord Community's profile picture LeRobot Worldwide Hackathon's profile picture Hugging Face MCP Course's profile picture Robotics Course's profile picture Hugging Science's profile picture Shirova AI's profile picture MCP-1st-Birthday's profile picture