Augmented Reality (AR)
Assembly Line
Palmer Luckey Is Bringing Anduril Smarts to Microsoft’s Military Headset
While known primarily for drones and air defenses, Anduril’s core offering is Lattice, a suite of software that powers those tools and a platform that can integrate with third-party systems. With today’s announcement, Lattice will be implemented in the Integrated Visual Augmentation System headset. Developed by Microsoft for the US military in 2021 and based on the company’s Hololens system, IVAS is an augmented-reality display that blends virtual information with a user’s view of the real world.
While virtual and augmented reality headsets remain a tough sell for the general public, Luckey says the US Army has been eager to experiment with the technology, primarily as a training tool but also as a way to deliver information more efficiently in critical situations. “The military is different,” Luckey says. “If you have an augmented-reality display that can make you 20 percent more lethal or make someone 10 percent safer, that’s a bigger improvement than just about any piece of gear you could give you.”
Northrop Grumman Uses Augmented Reality to Assemble Satellites
Augmented reality (AR) technology is a key ingredient of Industry 4.0 and digital manufacturing initiatives. It brings components of the digital world into a person’s perception of reality. AR layers computer-generated imagery onto a user’s view of the real world, providing a composite view.
Northrop Grumman Corp. is one of a handful of leading manufacturers that are using wearable devices to empower assemblers. The company recently invested in AR technology to streamline operations at its historic Space Park facility in Redondo Beach, CA.
“Augmented reality is a technology that takes our view of the real world and overlays useful and relevant digital data on top of that view,” says Oscar Castillo, a mechanical engineer who serves as factory modernization and digital transformation project manager at Space Park. “This is usually achieved through a smartphone, or in recent years through an AR headset such as Microsoft’s HoloLens.
“We have found that virtual reality can help our design teams assess a product’s producibility from a human factors or ergonomics perspective,” says Castillo. “Using engineering CAD models, we can spot challenges early enough in the design cycle so that changes can be made without great cost or schedule impact.
According to Castillo, AR technology is ideal for a variety of manufacturing applications. Most of the products built at Space Park have a 3D CAD model associated with them. Northrop Grumman engineers are leveraging those models to create augmented reality work instructions (ARWI).
Squint in Action: Responding to Overpressure Events at Eastern Gas Transmission, a BHE company
Smart Glasses Aid Inspection of Automotive Wire Harnesses
Wearable image acquisition devices are a better option, due to their better flexibility and adaptability. To that end, we have developed smart glasses that use machine vision, artificial intelligence and augmented reality to help assemblers inspect wire harnesses. Our system is specifically designed to help assemblers check various parts of a harness, including the serial number, relay box labeling, relay types, the number of relays, and the size and color of the relays, focusing on aspects such as alphanumeric characters, colors and shapes.
For our system, we chose YOLOv5s AI-based vision software from Ultralytics Inc. YOLOv5 is in the You Only Look Once (YOLO) family of computer vision models. It is commonly used for detecting objects. YOLOv5 comes in four main versions: small (s), medium (m), large (l), and extra large (x), each offering progressively higher accuracy rates. Each variant also takes a different amount of time to train.
Manufacturing and Transportation Industry Forum | DAIS 2024
SK Innovation’s Ulsan plant goes smarter with AI, robot dog, AR tech
At a petrochemical plant run by SK Innovation Co., South Korea’s largest oil refiner, Spot, a robot dog, was on routine patrol around the factory to spot any potential gas leaks. The $131,600 Spot, made by US robotics startup Boston Dynamics, recently joined SK Innovation, the parent of SK Energy Co., as its robot employee to enhance safety in SK’s plant operations. At another plant where a new building is under construction, SK Energy officials were checking a smart scaffolding system developed with augmented reality (AR) technology for efficient and safe construction.
In February, SK Energy teamed up with PTC Korea Co. to jointly enter the global smart plant construction business. Under their partnership, SK plans to use PTC Korea’s software technology in its next-generation facility management system OCEAN-H, or optimized & connected enterprise asset network hub. OCEAN-H, a system that systematically accumulates data on energy and chemical industry facilities within factories, is used to improve efficiency and safety in plant operations.
SIGMA: An open-source mixed-reality system for research on physical task assistance
What would it take to build an interactive AI system that could assist you with any task in the physical world, just as a real-time expert would? To begin exploring the core competencies that such a system would require, we developed and released the Situated Interactive Guidance, Monitoring, and Assistance (SIGMA) system, an open-source research platform and testbed prototype for studying mixed-reality task assistance. SIGMA provides a basis for researchers to explore, understand, and develop the capabilities required to enable in-stream task assistance in the physical world.
Large Part Kitting and Sequencing Guided by AR
In this video, discover how LightGuide projected augmented reality (AR) revolutionizes large-scale part kitting and sequencing processes. Watch as the operator is seamlessly guided through each step with intuitive AR work instructions, from precise placement of the parts cart to accurate selection and placement of components using a handheld barcode scanner integrated with the MES. Experience how LightGuide ensures every part is in its proper sequence, enhancing efficiency and accuracy in your operations
Lumus and SCHOTT Strengthen Manufacturing Partnership to Meet the Growing Global Market Demand for Optical AR Glasses
Lumus, the pioneering developer of reflective waveguide technology for augmented reality (AR) eyewear, announced today that it is expanding its five-year long partnership with SCHOTT, international technology group and manufacturer of high-tech materials and optics including SCHOTT RealView® high-index wafers and reflective waveguides, to meet the growing worldwide demand for optical AR glasses.
In the first phase of the five-year partnership, SCHOTT established a sampling line in Malaysia, where it successfully developed the processing and metrology capabilities to provide customers with prototypes and better understand their manufacturing requirements. In the second phase, SCHOTT focused on developing the equipment and processing platform to meet the stringent requirements of industrial manufacturing for these customers. The engine of this processing platform is Lumus’s new Z-lens reflective waveguide architecture that’s designed to meet the imminent customer demand for AR glasses.
Augmented reality makes new robots easier to start up
New KUKA.MixedReality software visualizes the environment of robot cells live on your smartphone to support fast, safe and intuitive robot start-up. The mobile app displays tools and interference geometries to enable early detection of potential hazards so users can eliminate them before a robot starts work.
Augmented Reality (AR) enables intuitive robot startup. It connects the real and virtual worlds to enrich the environment of the robotic cell with clear, uncomplicated digital information. Users can detect and correct errors quickly, which accelerates installation and increases safety. For example, the software can simulate robot motion with a virtual gripper. To prevent damage to the robot or gripper, any potential collisions that show up in the AR environment can be prevented early in the real environment.
The software can be used to simulate robot movement with a virtual gripper, for example. If potential collisions are detected in the AR environment, they can be prevented at an early stage in the real environment so that neither the robot nor the gripper is damaged. KUKA.MixedReality consists of the KUKA.MixedReality Assistant app and the additional KUKA.MixedReality Safe technology package, which is installed on the robot controller.
Connected Digital Manufacturing: Cobots and Augmented Reality for Electronics Assembly
LightGuide’s industrial augmented reality (AR) work instruction platform seamlessly integrates with a wide variety of digital manufacturing technologies, factory tools, and IO devices, including cobots. Here, LightGuide is integrated with KUKA’s LBR iisy cobot to combine the benefits of industrial automation and digital work instructions to streamline the process of assembling an electrical component.
Hitachi's metaverse tech helps keep Japanese craftsmanship alive
Hitachi has developed a metaverse-based system that lets skilled workers run remote demonstrations using real-world factory equipment, with an eye toward industries where know-how is concentrated among aging veterans. This reporter experienced it firsthand with demonstration equipment. Through a virtual reality headset, an image of an instructor’s hands was superimposed on a drum-shaped machine.
Warping in the real world: Remote factory meetings in 360°
In a research project with SB&D, ETH students Michael Stegmaier and Cesare Primultini led our field test of interactive mixed reality live streaming technology in manufacturing. Michael and Cesare conceptualized two ways to transmit live-streaming 360° video from the shop floor of SB&D: A first-person view where a camera was mounted to a hard hat and a fixed-position view where the camera could be mounted inside a machine during a changeover. Live-streamed 360° video would be transmitted to a cloud, where participants would meet virtually from anywhere (see figure below). Back in 2021, we identified Avatour, a startup from California, as our project’s only viable software provider.
Where can this technology find good use? High-potential use cases are safety audits, training programs, and tasks that require the presence of an expert or coach. Other use cases are factory tours, external or internal audits, technology transfer projects, inspections, and on-the-job coaching of individuals.
How Prominent Video Game Technology is Changing Industrial Training and Manufacturing
Panigrahi explains that GridRaster’s XR equipment is currently being used in the manufacturing of airplane wings, spacecraft parts and automobile components, adding that AR and VR have emerged as a crucial part of repair maintenance for these sectors. “Having a headset that can give you instructions, workflows overlayed on top of the asset you are fixing, and working through each and every instruction has been not only a time-saver but a life-saver as well because you’re ensuring those human errors can be completely eliminated,” Panigrahi says.
🤝 PTC and Rockwell Automation Extend Relationship with Focus on IoT and Augmented Reality for Manufacturing
PTC (NASDAQ PTC) and Rockwell Automation (NYSE ROK) today announced an extension of their relationship, focused on the adoption of Internet of Things (IoT) and augmented reality (AR) software by manufacturing companies. Rockwell Automation will continue to resell PTC’s ThingWorx® IoT software, including the Digital Performance Management (DPM)™ manufacturing solution, and Vuforia® AR software to new and existing customers across discrete and process manufacturing verticals. Customers will continue to benefit from both companies’ resources, technologies, and industry expertise.
Eliminating Defects with AR Technology
L3Harris Technologies implemented LightGuide AR software to ensure standardization across a variety of complex processes. Following the implementation of LightGuide on a line with 17 variants of one product, visually guided workflows helped consolidate parts, which eliminated changeover per variant and resulted in zero assembly-related defects. Since these results, L3Harris has implemented LightGuide on complex manual lines at multiple locations.
According to one Engineering Manager at the L3Harris, the system uses infrared and 3D sensing to know where an operator’s hand is within an inch in any direction. The cues highlighting what pieces go where allow operators to focus on the task at hand, not where they are in the process; further, the system will notify the operator via visual cues and messaging and stop instruction if they skip a step or reach for the wrong component.
AR in Manufacturing Industry
The manufacturing sector’s usage of augmented reality (AR) is the main topic of this blog. By superimposing digital information over the real world, augmented reality (AR) technology gives workers access to real-time information and direction. Design and prototyping, assembly and production, maintenance and repair, as well as training and teaching, are just a few of the manufacturing-related uses for augmented reality technology. AR may improve productivity, improve safety, and reduce costs in manufacturing operations, making it an attractive technology for organizations in the industry. The initial expenditure in hardware and software, as well as ongoing maintenance and updates, might make integrating AR challenging.
How Smart Glasses Helped a Manufacturer Through the Pandemic
How does an expert in one facility guide an engineer halfway across the country through implementing a new technology? Food and beverage packaging manufacturer Crown Holdings found an innovative answer during the pandemic, when its employees couldn’t travel freely: it equipped them with smart glasses.
“When the pandemic started, we were trying to coordinate commissioning activities through email and WhatsApp messaging,” said Crown Holdings Project Industrial Engineer Leon Azzi. “Tasks that normally took two to three days were taking weeks.” But with the glasses, “[The workers] could share with each other the PLC (programmable logic controllers) electrical diagrams and pictures using the glasses viewer, and the remote experts could point them to areas to focus on in real time,” said Crown Holdings Digital Optimization Team Head Alberto Rodriguez.
Reality Show: X-ray Vision Can See Through Metal
A typical aircraft maintenance inspection involves maintenance technicians and engineers walking around an aircraft recording new defects and damage with a pencil in a notebook. Locations are often described in language like ‘3 inches from the left side of the window.’ The inspection can often take hours or days. But what if you could hold a digital device and see locations of all previous damage and repairs highlighted in 3D?
Young Guns shoot for AI automated shipping container
“There is a significant drive from leading retailers internationally to innovate within their supply chain transformation strategies – predictability, visibility, and productivity will be a key component in these transformations, and we believe our shipping container innovation will create opportunities for this to occur at the container face that are not currently availability in the industry.”
The team mocked up their distribution centre, containers, and the Polaris product in a 3D CAD (computer-aided design) model, and transitioned the data into an augmented reality environment in HoloLens. The simulation used in the development of Polaris tracks the actions and interaction of each aspect of container de-stuffing. Each unpacker, forklift, container, and pallet are represented as an agent within a distribution centre. “We showed what the product would look like at various stages of utilisation in the distribution centre,” Kelly said. “We had YGCC’s frontline team come in and they could put the headsets on, and they could walk around in the virtual future distribution centre environment with two people at a time. “They could discuss what it would be like to use the device in different stages and get a feel for it and what it’s trying to do. That was hugely beneficial for them in terms of being able to engage with what the device is, because they didn’t just have to imagine being with it, they could actually be with it.”
“Waste time in the form of movement and waiting can be reduced from 25 down to 5 per cent,” Mr Kelly said. “Allocating the correct number of workers per stock type is important in achieving these improvements.”
HoloLens 2 helps Novo Nordisk employees see work in new ways
The pharmaceutical giant, a leader in the global production of diabetes medicine, has been pioneering the use of emerging digital technologies for everything from simplifying floor operations in their manufacturing facilities across the globe to conducting virtual site visits from thousands of miles away. Some of those solutions include Microsoft HoloLens 2, Microsoft Dynamics 365 Remote Assist, Microsoft Dynamics 365 Guides and Microsoft Power Platform. This includes introducing what is known as augmented instructions, via HoloLens 2, to shop floor employees, providing them with new possibilities to support both process performance and getting onboarded for new procedures.
Kawasaki Heavy Industries (Satya Nadella 2022 Build Keynote)
Flexible robotic arm put to work with AR
According to Imperial, the flexible arm can twist and turn in all directions, making it customisable for applications in manufacturing, spacecraft maintenance, and injury rehabilitation. In use, people working with the robot would manually bend the arm into the precise shape needed for each task, a level of flexibility made possible by layers of mylar sheets inside, which slide over one another and can lock into place. So far, configuring the robot into specific shapes without guidance has presented challenges.
To enhance the robot’s user-friendliness, researchers at Imperial’s REDS (Robotic manipulation: Engineering, Design, and Science) Lab designed a system for users to see in AR how to configure their robot. Wearing mixed reality smartglasses and through motion tracking cameras, users see templates and designs in front of them superimposed onto their real-world environment. They then adjust the robotic arm until it matches the template, which turns green on successful configuration so that the robot can be locked into place.
Semi-virtual site visits deliver enhanced customer value
AR technology offers opportunities to schedule visits that have been difficult to arrange in the past due to sites being in remote locations, offshore environments or other restricted areas. Headset technology allows field engineers to reach someone where they are and streamline the visit process. As a result, someone can receive insights in real time because remote engineers can see what is happening onsite versus trying to identify issues through messages or images (see Figure 4). Problems can be solved more quickly, which saves time and money in the process.
Gemini - Collaboration using Augmented Reality
Our New Reality: The Rise of AR and VR
“The ability to use digital twins and simulation in AR [and] VR is gaining steam,” says Kelly Malone of Taqtile. “New technologies like 5G and edge computing enable new use cases where access to data-intensive operations is feasible in-situ using HMDs and mobiles devices. So rather than minutes or hours to see the impact of a design change, these results can be viewed in-situ within minutes or seconds to support the evaluation of more design options and iterations. Sensor information (industrial internet of things – IIoT) can also be integrated and overlaid upon the real-world or digital twin, providing an added layer of data to inform design change impacts and decision making.”
This Factory Is Using AR To Help With A Hiring Crunch
One of the challenges associated with AR has been in trying to turn a complex physical process, such as wiring a component or working a machine, into code that could run on a headset. Taqtile CEO Dirck Schou said the company’s software makes programming for AR glasses simple, and based on my conversation with Tim Lecrone and Beau Wileman of PBC, the software Taqtile developed is easy to use. Once PBC has created a module for training it pays for itself after 1.44 employees train with it according to Wileman.
The cobots help handle processes that are repetitive and free up people to take on different tasks. Given how tough it is to hire people to work in the factory, using them helps reduce the overall staffing load. But the biggest gains so far have been in training and getting employees quickly up to speed. Now PBC can hire a person and get them working on a machine in a few days as opposed to that taking up to six weeks. It also helps reduce the cost of training a cobot and staff. Wileman told me that an intern, which costs $17 an hour, can train a cobot or map out a process in less than four hours, while it might cost around $30,000 for an outside expert to manually train a cobot.
CEOs Are Dooming Business Travel — Maybe for Good
Take Akzo Nobel NV, Europe’s biggest paint maker, for instance. At its Amsterdam headquarters, Chief Executive Officer Thierry Vanlancker has spent the past year watching his manufacturing head, David Prinselaar, flap his arms, madly gesticulate and seemingly talk to himself while “visiting” 124 plants by directing staff with high-definition augmented-reality headgear on factory floors. A task that meant crisscrossing the globe in a plane before is now done in a fraction of the time — and with no jet lag. For Vanlancker, there’s no going back.
Royal Dutch Shell Plc has created online control rooms with interactive 3D simulations of oil platforms and plants, giving engineers virtual access from home. In Troy, Michigan, Kevin Clark, the CEO of Aptiv Plc, a former car parts unit of General Motors Co., is using drones and Oculus augmented-reality headsets to show customers the performance and manufacturing run rates of plants in Mexico, Hungary, or China.
Industry 4.0 and the Automotive Industry
“It takes about 30 hours to manufacture a vehicle. During that time, each car generates massive amounts of data,” points out Robert Engelhorn, director of the Munich plant. “With the help of artificial intelligence and smart data analytics, we can use this data to manage and analyze our production intelligently. AI is helping us to streamline our manufacturing even further and ensure premium quality for every customer. It also saves our employees from having to do monotonous, repetitive tasks.”
One part of the plant that is already seeing benefits from AI is the press shop, which turns more than 30,000 sheet metal blanks a day into body parts for vehicles. Each blank is given a laser code at the start of production so the body part can be clearly identified throughout the manufacturing process. This code is picked up by BMW’s iQ Press system, which records material and process parameters, such as the thickness of the metal and oil layer, and the temperature and speed of the presses. These parameters are related to the quality of the parts produced.
How Augmented Reality Strengthens Biotech Manufacturing
Probably, the biggest advantage of AR is it enables seeing the production process virtually, without the need to be there. “It’s a game-changer for the industry. Individuals can have eyes and ears on site at a moment’s notice to address an emerging issue, or to host routine remote collaboration sessions,” Stracquatanio highlights.
AR can also increase control over the manufacturing process. Pharma and biotech companies cannot afford mistakes during the production phase. A little oversight might lead to serious consequences such as having to start from scratch, which can be very expensive and time-consuming. A recent example is that of Johnson & Johnson’s manufacturing partner Emergent BioSolutions, whose workers erroneously mixed ingredients from two different Covid-19 vaccines; this led to wasting around 15 million vaccine doses.
During a working day, we could see an operator who loads 3D models of biotech instruments, looking at specific pieces and relevant information appearing in the smart glasses or tablet. Meanwhile, another engineer walks up to a machine with a QR code, and instructions pop up in the glasses, facilitating access to the adjustments. A few steps from there, another colleague is looking at batch records, saving values into the system just via voice.
Augmented reality becomes actual reality
When applied to electrical power distribution across a wide range of businesses and industries, AR has the potential to greatly increase power availability, electrical safety, and efficiency. Here’s why:
- Availability: AR helps organizations optimize operations and maximize continuity for better productivity and profitability
- Safety: AR helps to reduce the risk of occupational injuries and fatalities
- Efficiency: AR help reduces the total cost of ownership by offering more accessible and effective training
How Augmented Reality Became a Serious Tool for Manufacturing
Making monsters appear in games like Pokémon Go is not the only application for augmented reality these days. Industry is using the technology too, harnessing CAD data for training workers, standardizing workflows, and enabling collaboration.
Augmented Reality Gets Pandemic Boost
Augmented reality, which superimposes digital content onto a user’s view of the real world, became more valuable for some companies such as Mercedes-Benz USA and L’Oréal SA last year amid social distancing requirements and lockdowns. The companies are using the technology to provide assistance for employees and consumers in real-time, without needing to be physically present.
Advanced Technologies Adoption and Use by U.S. Firms: Evidence from the Annual Business Survey
While robots are usually singled out as a key technology in studies of automation, the overall diffusion of robotics use and testing is very low across firms in the U.S. The use rate is only 1.3% and the testing rate is 0.3%. These levels correspond relatively closely with patterns found in the robotics expenditure question in the 2018 ASM. Robots are primarily concentrated in large, manufacturing firms. The distribution of robots among firms is highly skewed, and the skewness in favor of larger firms can have a disproportionate effect on the economy that is otherwise not obvious from the relatively low overall diffusion rate of robots. The least-used technologies are RFID (1.1%), Augmented Reality (0.8%), and Automated Vehicles (0.8%). Looking at the pairwise adoption of these technologies in Table 14, we find that use of Machine Learning and Machine Vision are most coincident. We find that use of Automated Guided Vehicles is closely associated with use of Augmented Reality, RFID, and Machine Vision.
Digitalisation at BASF: HoloLens
Boeing Tests Augmented Reality in the Factory
Installing electrical wiring on an aircraft is a complex task that leaves zero room for error. That’s why Boeing is testing augmented reality as a possible solution to give technicians real-time, hands-free, interactive 3D wiring diagrams - right before their eyes.
“Our theory studies have shown a 90 percent improvement in first-time quality when compared to using two-dimensional information on the airplane, along with a 30 percent reduction in time spent doing a job.”
Google Glass Didn't Disappear. You Can Find It On The Factory Floor
With Google Glass, she scans the serial number on the part she’s working on. This brings up manuals, photos or videos she may need. She can tap the side of headset or say “OK Glass” and use voice commands to leave notes for the next shift worker.
Peggy Gullick, business process improvement director with AGCO, says the addition of Google Glass has been “a total game changer.” Quality checks are now 20 percent faster, she says, and it’s also helpful for on-the-job training of new employees. Before this, workers used tablets.
Augmented Reality Is Already Improving Worker Performance
The video below, for example, shows a side-by-side time-lapse comparison of a GE technician wiring a wind turbine’s control box using the company’s current process, and then doing the same task while guided by line-of-sight instructions overlaid on the job by an AR headset. The device improved the worker’s performance by 34% on first use.
There’s been concern about machines replacing human workers, and certainly this is happening for some jobs. But the experience at General Electric and other industrial firms shows that, for many jobs, combinations of humans and machines outperform either working alone. Wearable augmented reality devices are especially powerful, as they deliver the right information at the right moment and in the ideal format, directly in workers’ line of sight, while leaving workers’ hands free so they can work without interruption. This dramatically reduces the time needed to complete a job because workers needn’t stop what they’re doing to flip through a paper manual or engage with a device or workstation. It also reduces errors because the AR display provides explicit guidance overlaid on the work being done, delivered on demand. Workers need only follow the detailed instructions directly in front of them in order to move through a sequence of steps to completion. If they encounter problems, they can launch training videos or connect by video with remote experts to share what they see through their smart glasses and get real-time assistance.