personalrobot.ai

#Personal Robot AI Meta

#Laser-based heart monitoring robot | Laser device detecting heartbeat, breathing rate, and muscle activity from up to meter away, without requiring any wires or contact

#Object localization

#Object tracking

#Visual servoing

#Spatially aware manipulation

#Hand-eye coordination

#Semantic scene understanding

#AI and machine learning

#3-D reconstruction

#Unstructured Robotics | Performing tasks not predetermined or predefined | Real-time perception

#Robotics Perception

#Human-assisted robotics

#Haptic technology

#Object recognition

#Machine learning

#Redundant degrees of freedom

#Avatar robot

#Remote avatar robot

#Physical movement

#Virtual movement

#Emotional movement

#GPT language

#Sensors tracking movement

#Face recognition

#Voice recognition

#Detecting emotions

#Detecting age

#Communicating common expressions like astonishment and surprise

#Communicating gestures like yawning and shrugging

#Artificial neural network

#Having ear for music

#Non Verbal Cue

#Machine Learning

#Gesture Control

#Surrounding Sensing

#Owner Learning

#Task Intelligence

#Health Monitoring

#Problem Detecting

#Death Management

#Humanoid robot

#Believable robotic character

#Dynamic robotic performance

#Actuated figure

#Untethered, high-velocity gymnastic action robustly and repeatably

#Stuntronics

#Socially intelligent machine

#Cognitive intelligence

#AI Interactions

#Human-like robot

#Inference Engine

#Conversational AI assistant

#Interactive co-pilot

#Token generation

#Models of muscle tissue

#Musculoskeletal health

#Cardiovascular health

#Action recognition model

#Neurological degeneration

#Neuromuscular junctions

#Synthetic data generation (SDG)

#Training

#Stimulation

#Teleradiology

#Autonomous mobility assistance

#Blind individuals

#Visually-impaired individuals

#Regain independence

#Geometric fabrics

#Trained policies

#Reforcement Learning (RL) | Does not rely on labeled data | Learning through trial and error using reward-punishment | Transferring simulation results to real robotic hardware remains challenge | Need to train policies that generate a variety of agile behavior on physical hardware to achieve novel, robust, and practical locomotion behavior | Robot capability to manipulate objects and fixtures, such as doors and levers, in conjunction with locomotion would significantly enhance its utility | Exploring full-body contact strategies | Exploring high-performance, whole-body locomotion and tasks requiring full-body contact strategies, such as dynamic running and full-body manipulation of heavy objects | Developing close coordination between arms and legs | Developing reinforcement learning to generate behavior during complex contact events without imposing strict requirements | Reinforcement learning algorithms for humanoid locomotion | Reinforcement learning algorithms for humanoid manipulation policies | Prototyping reinforcement learning algorithms | Pybullet | IssacSim | Mujoco | Training RL policies in simulation

#Context length

#Tokens/s speed

#AI infrastructure

#AI inference technology

#Digital Twin of Person

#Retrieval-Augmented Generation (RAG) | Enhancing large language models (LLMs) by integrating external knowledge sources | Improving accuracy and relevance of generated responses | Allows LLMs to retrieve pertinent information from databases or documents before generating answers | Reducing errors and hallucinations typical in traditional models

#SLAM | Simultaneous Localization and Mapping

#Body analysis

#Facial expression analysis

#Eye gaze control

#Face segmentation

#CRUD | Create, Read, Update, and Delete | Four fundamental operations for managing persistent data in applications

#Ubiquitous

#AIoT space

#Electric actuator | Device converting electrical energy into mechanical motion | Using electric motor to create movement or apply force

#Legged robot

#Multimodal access

#Electromyography (EMG)

#Spatial control

#Dexterity

#Bionics

#Exoskeleton

#Cogging torque

#Cobot

#Soft robot

#Pneumatic device

#Soft silicone

#Pneumatic channel

#Agentic workflow

#Learning Management System (LMS)

#Time To First Token (TTFT)

#Build simulation stack

#Close sim-to-real gap

#Building contact-rich physics simulation

#Scaling up reinforcement learning algorithms

#Drive AI model evaluation and iteration speed

#Build ML robotics simulation

#Work across AI stack

#Build data engine to log, clean, label data for foundation model training

#Pybullet

#IssacSim

#Mujoco

#Blender

#Maya

#GPU simulation

#NoSQL

#Python | Testing large codebases in Python

#Deep Learning Frameworks | Pytorc | TF | JAX

#Managing ML compute clusters

#Linear algebra

#Supervised machine learning

#Robot Opersting System | ROS | ROS2

#Open source LLM technology

#Simulating and validating robots, for multiple domains

#Extracting key sction points

#Skeletal data

#Scene

#Character assets

#Asset path

#Programming new action for subject

#Character primitive

#Re-rendering stage

#Spatial-temporal graph convolutional network (ST-GCN)

#3D skeleton data

#Action recognition

#Zero-shot inferencing

#Artificial General Intelligence (AGI) | AI capable of understanding, learning, and performing any intellectual task that a human can

#Core knowledge systems | Either innate or developed early on in humans and some non-human animals

#Objectness | Knowledge that world can be parsed into objects that have certain physical properties

#Numerosity | Knowledge of small quantities | Notions of smallest, argest, greater than, less than

#Basic geometry and topology | Knowledge of lines, simple shapes, symmetries, containment, and copying

#Agents and goals | Knowledge some entities are agents who have their own intentions and act to achieve goals

#Few-shot learning | Each task has only a few examples from which an abstract rule must be inferred

#Text-only language model

#Multimodal foundation model | Able to deal with both text and images

#Polymathic AI | Developing machine learning models integratung knowledge across various scientific disciplines | Training AI with diverse datasets | Providing open-source training datasets | Fostering collaboration among researchers | Creating foundation models to identify commonalities across disciplines

#Robot Teaching Method using Hand Gestures and Poses

#Programming robots by human demonstration

#Convolution Neural Network (CNN) to recognize gestures

#Robot motion primitives

#Managing primitives in robot system

#Behaviour-based programming platform

#Extensible Agent Behavior Specification Language (XABSL)

#Hand motion sequence

#Robot pick-and-place task

#Human-Robot interaction (HRI)

#Understanding human intention through human behaviours

#Hand gesture recognition system

#Training database

#Gesture modelling

#Task goal

#Behaviour-based robot system

#Gesture command

#Robot task sequence

#CNN-based gesture recognition system

#Multi-tasking scheduling to process images from cameras and robot encoders, and command robot to behave under proper conditions

#7 Degrees of Freedom (7-DOF) | Offers kinematic redundancy | Allows multiple joint configurations to achieve the same end-effector pose | Obstacle Avoidance: extra DOF helps avoid collisions with obstacles or the robot itself | Singularity Avoidance: minimizes gimbal lock and improves motion flexibility in constrained environments | Enhanced Manipulability: allows better control over speed, strength, and precision | Expanded Workspace: 7-DOF arm can operate effectively in more complex environments compared to 6-DOF systems

#California wildfire | Challenges | Access roads too steep for fire department equipment | Brush fires | Dangerously strong winds for fire fighting planes | Drone interfering with wildfire response hit plane | Dry conditions fueled fires | Dry vegetation primed to burn | Faults on the power grid | Fires fueled by hurricane-force winds | Fire hydrants gone dry | Fast moving flames | Hilly areas | Increasing fire size, frequency, and susceptibility to beetle outbreaks and drought driven mortality | Keeping native biodiversity | Looting | Low water pressure | Managing forests, woodlands, shrublands, and grasslands for broad ecological and societal benefits | Power shutoffs | Ramping up security in areas that have been evacuated | Recoving the remains of people killed | Retardant drop pointless due to heavy winds | Smoke filled canyons | Santa Ana winds | Time it takes for water-dropping helicopter to arrive | Tree limbs hitting electrical wires | Use of air tankers is costly and increasingly ineffective | Utilities sensor network outdated | Water supply systems not built for wildfires on large scale | Wire fault causes a spark | Wires hitting one another | Assets | California National Guard | Curfews | Evacuation bags | Firefighters | Firefighting helicopter | Fire maps | Evacuation zones | Feeding centers | Heavy-lift helicopter | LiDAR technology to create detailed 3D maps of high-risk areas | LAFD (Los Angeles Fire Department) | Los Angeles County Sheriff Department | Los Angeles County Medical Examiner | National Oceanic and Atmospheric Administration | Recycled water irrigation reservoirs | Satellites for wildfire detection | Sensor network of LAFD | Smoke forecast | Statistics | Beachfront properties destroyed | Death tol | Damage | Economic losses | Expansion of non-native, invasive species | Loss of native vegetation | Structures (home, multifamily residence, outbuilding, vehicle) damaged | California wildfire actions | Animals relocated | Financial recovery programs | Efforts toward wildfire resilience | Evacuation orders | Evacuation warnings | Helicopters dropped water on evacuation routes to help residents escape | Reevaluating wildfire risk management | Schools closed | Schools to be inspected and cleaned outside and in, and their filters must be changed

#Quadruped robot | Lour-legged robot | Mimic animal locomotion | Navigate rough, uneven, cluttered terrains | Climb stairs | Operate indoors or outdoors

#Cognitive AI

#Athletic AI

#A-list celebrity home protector | Burglaries targeting high-end items | Burglary report on Lime Orchard Road | Burglar had smashed glass door of residence | Ransacked home and fled | Couple were not home at the time | Unknown whether any items were taken | Lime Orchard Road is within Hidden Valley gated community of Los Angeles in Beverly Hills | Penelope Cruz, Cameron Diaz, Jennifer Lawrence, Adele and Katy Perry have purchased homes there, in addition to Kidman and Urban | Kidman and Urban bought their home for $4.7 million in 2008 | 4,100-square-foot, five-bedroom home built in 1965 and sits on 1¼-acre lot | Property large windows have views of the canyons | Theirs is one of several celebrity properties burglarized in Los Angeles and across country recently | Connected to South American organized-theft rings

#Professional athlete home protector | South American crime rings | Targeting wealthy Southern California neighborhoods for sophisticated home burglaries | Behind burglaries at homes of professional athletes and celebrities | Theft groups conduct extensive research before plotting burglaries | Monitoring target whereabouts and weekly routines via social media | Tracking travel and schedules | Conducting physical surveillance at homes | Attacks staged while targets and their families are away | Robbers aware of where valuables are stored in homes prior to staging break-ins | Burglaries conducted in short amount of time | Bypass alarm systems | Use Wi-Fi jammers to block Wi-Fi connections | Disable devices | Cover security cameras | Obfuscate identities

#Dexterous robot | Manipulate objects with precision, adaptability, and efficiency | Dexterity involves fine motor control, coordination, ability to handle a wide range of tasks, often in unstructured environments | Key aspects of robot dexterity include grip, manipulation, tactile sensitivity, agility, and coordination | Robot dexterity is crucial in: manufacturing, healthcare, logistics | Dexterity enables automation in tasks that traditionally require human-like precision

#Agentic AI | Artificial intelligence systems with a degree of autonomy, enabling them to make decisions, take actions, and learn from experiences to achieve specific goals, often with minimal human intervention | Agentic AI systems are designed to operate independently, unlike traditional AI models that rely on predefined instructions or prompts | Reinforcement learning (RL) | Deep neural network (DNN) | Multi-agent system (MAS) | Goal-setting algorithm | Adaptive learning algorithm | Agentic agents focus on autonomy and real-time decision-making in complex scenarios | Ability to determine intent and outcome of processes | Planning and adapting to changes | Ability to self-refine and update instructions without outside intervention | Full autonomy requires creativity and ability to anticipate changing needs before they occur proactively | Agentic AI benefits Industry 4.0 facilities monitoring machinery in real time, predicting failures, scheduling maintenance, reducing downtime, and optimizing asset availability, enabling continuous process optimization, minimizing waste, and enhancing operational efficiency

#Field Foundation Model (FFMs) | Physical world model using sensor data as an input | Field AI robots can understand how to move in world, rather than just where to move | Very heavy probabilistic modeling | World modeling becomes by-product of Field AI.robots operating in the world rather than prerequisite for that operation | Aim is to just deploy robot, with no training time needed | Autonomous robotic systems applucations | Field AI is software company making sensor payloads that integrate with their autonomy software | Autonomous humanoid Field AI can do | Focus on platforms that are more affordable | Integrating mobility with high-level planning, decision making, and mission execution | Potential to take advantage of relatively inexpensive robots is what is going to make the biggest difference toward Field AI commercial success

#Multipurpose commercial humanoid | Potential for useful and reliable and affordable humanoids | Difficult problem making highly technical piece of hardware and software compete effectively with humans in labor market | Robots are not hard to build; but they are hard to make useful and make money with | Whole perception pipeline running at the framerate of sensors nowadays | All the technology is here now | Starting with surrogate robot from someone else to get autonomy team going while building own robot in parallel | Giving out a significant chunk of the company to early joiners | Combined efforts of the research community enables commercialization | Building team is really important

#Humanoid robots and fashion future | Shanghai, humanoid robots transcend fashion hype, reimagining design, challenging beauty norms, and unlocking metaverse opportunities | Convergence of fashion and technology | Human-machine collaboration in fashion | Genuine, emerging trend | Creativity, production, and human-machine interaction | Robots are becoming experimental platforms | Integration of robots into runway | Aesthetic Reinvention: designing beyond the human form | Fostering Human-Robot Collaboration From Runway to Production and Retail | Challenging Beauty Norms | Paving Way for Future Trajectories: The Metaverse of Fashion

#Large Language Model (LLM) | Foundational LLM: ex Wikipedia in all its languages fed to LLM one word at a time | LLM is trained to predict the next word most likely to appear in that context | LLM intellugence is based on its ability to predict what comes next in a sentence | LLMs are amazing artifacts, containing a model of all of language, on a scale no human could conceive or visualize | LLMs do not apply any value to information, or truthfulness of sentences and paragraphs they have learned to produce | LLMs are powerful pattern-matching machines but lack human-like understanding, common sense, or ethical reasoning | LLMs produce merely a statistically probable sequence of words based on their training | LLMs are very good at summarizing | Inappropriate use of LLMs as search engines has produced lots of unhappy results | LLM output follows path of most likely words and assembles them into sentences | Pathological liars as a source for information | Incredibly good at turning pre-existing information into words | Give them facts and let them explain or impart them

#Retrieval Augmented Generation. (RAG LLM) | Designed for answering queries in a specific subject, for example, how to operate a particular appliance, tool, or type of machinery | LLM takes as much textual information about subject, user manuals and then pre-process it into small chunks containing few specific facts | When user asks question, software system identifies chunk of text which is most likely to contain answer | Question and answer are then fed to LLM, which generates human-language answer in response to query | Enforcing factualness on LLMs

#Mobile robotics | Mobile robots that move automatically on wheels | Making automation mobile | Cameras and lasers can be installed in mobile robots

#Unitree R1 humanoid | Agile mobility: 24-26-DOF for adaptation to complex scenarios; its 2-DOF head enhances environmental perception | Lightweight structure, easy maintenance: ≤121cm agile form, ultra-lightweight at about 25kg, ready out-of-the-box to empower | Integrated with Large Multimodal Model for voice and images: Fully open control interfaces for joints and sensors, with support for mainstream simulation platforms | Height Width and Thickness（Stand): 1210x357x190mm | Degree of Freedom(Total Joints): 24 | Single Leg Degrees of Freedom: 6 | Single Arm Degrees of Freedom: 5 | Waist Degrees of Freedom: 2 | Head Degrees of Freedom: None | Dexterous Hand: NOT | Joint output bearing: Crossed roller bearings, Double Hook Ball Bearings | Joint motor: Low inertia high-speed internal rotor PMSM(permanent magnet synchronous motor,better response speed and heat dissipation) | Maximum Torque of Arm Joint: 约 2kg | Calf + Thigh Length: 675 | Forearm + Upper Arm Length: 435 | Joint Movement Space: Waist Joint：Y±150° R±30°, Knee Joint：-10°~+148°, Hip Joint：Y:±157° P:-168° ~+146° R:-60° ~+100° | Electrical Routing: Hollow + Internal Routing | Joint Encoder: Dual + single encoder | Cooling System: Local air cooling | Power Supply: Lithium battery | Basic Computing Power: 8-core high-performance CPU | Microphone Array: 4-Mic Array | Speaker: YES | WiFi 6 | Bluetooth 5.2 | Humanoid Binocular Camera | NVIDIA Jetson Orin Optional (40-100 Tops) | Smart Battery (Quick Release) | Charger | Manual Controller | Battery Life: about 1h | Upgraded Intelligency: OTA | Warranty Period: 8 Months

#Large Behavior Model (LBM) | Controlling the entire robot actions | Joint research partnership between Boston Dynamics and Toyota Research Institute | Collaboration aims to create a general-purpose humanoid assistant | Whole-body movements: walking, crouching, and lifting to complete tasks that involve sorting and packing

#AI generalist robot | Developing end-to-end language-conditioned policies | Taking full advantage of capabilities of humanoid form factor, including taking steps, precisely positioning its feet, crouching, shifting its center of mass, and avoiding self-collisions | Building policies process: 1. Collect embodied behavior data using teleoperation on both real-robot hardware and in simulation, 2. Process, annotate, and curate data to easily incorporate it into machine learning pipeline, 3. Train neural-network policy using all of the data across all tasks | 4. Evaluate the policy using a test suite of tasks | Policy maps inputs consist of images, proprioception, language prompts to actions that control robot at 30Hz | Leveraging diffusion transformer together with flow matching loss to train model | Dexterous manipulation including part picking, regrasping | Subtasks triggered by passing a high-level language prompt to the policy | Reacting intelligently when things go wrong | With Large Behavior Model (LBM), training process is the same whether it is stacking rigid blocks or folding a t-shirt: if you can demonstrate it, robot can learn it | Speeding up the execution at inference time without requiring any training time changes

#Teleoperation | High-Quality Data Collection for Model Training | Control system allows to perform precise manipulation while maintaining balance and avoiding self-collisions | VR headset for operators to fully immerse themselves in the robot workspace and have access to the same information as the policy, with spatial awareness bolstered by a stereoscopic view rendered using head mounted cameras reprojected to the user viewpoint | Custom VR software provides teleoperator with a rich interface to command robot, providing them real-time feeds of robot state, control targets, sensor readings, tactile feedback, and system state via augmented reality, controller haptics, and heads-up display elements | One-to-one mapping between user and robot (i.e. moving your hand 1cm would cause robot to also move by 1cm) | To support mobile manipulation, tracking on feet added and teleoperation control extended to support stance mode, support polygon, and stepping intent to match that of operator

#Policy | Toyota Research Institute.Large Behavior Model | Diffusion Policy-like architecture | Boston Dynamic policy | Diffusion Transformer-based architecture | Flow-matching objective | Conditioned on proprioception, images | Accepting language prompt that specifies objective to robot | Image data comes in at 30 Hz | Network uses a history of observations to predict an action-chunk | Observation space consists of images from robot head-mounted cameras along with proprioception | Action space includes joint positions for left and right grippers, neck yaw, torso pose, left and right hand pose, and left and right foot poses | Shared hardware and software across two robots aids in training multi-embodiment policies that can function across both platforms, allowing to pool data from both embodiments | Quality assurance tooling allows to review, filter, and provide feedback on data collected

#Simulation | Allows to quickly iterate on teleoperation system and write unit and integration tests | Performing informative training and evaluations that would otherwise be slower, more expensive and difficult to perform repeatably on hardware | Simulation stack is faithful representation of hardware and on-robot software stack | Ability to share data pipeline, visualization tools, training code, VR software and interfaces across both simulation and hardware platforms | Benchmarking policy and architecture choices | Incorporating simulation as a significant co-training data source for multi-task and multi-embodiment policies deployed on hardware

#Vision-language model (VLM) | Training vision models when labeled data unavailable | Techniques enabling robots to determine appropriate actions in novel situations | LLMs used as visual reasoning coordinators | Using multiple task-specific models