Footwear Detection Image

Footwear defect detection using Vision Intelligence Systems

Across global industries, AI-driven computer vision systems are enabling the streamlining of the production process so that the products are compliant with the quality criteria set by the company.  This in turn brings in advantages of greater efficiency, lower operational costs while enabling 24/7 production and quicker decision making. 

Producing defect-free footwear is not easy. With a variety of defects to be recognized like excessive glue, weak bonds, scuff marks, asymmetry, size, metal contamination and sharp points, the time and effort for manual inspection keeps multiplying. Hence, the need for an automated defect detection system in the footwear industry.

This blog describes an application designed for defect detection in footwear raw materials using TYQ-i, Ignitarium’s Deep Learning based Vision Analytics Platform.  The project was executed for one of India’s leading footwear manufacturing companies to cut down on the time taken for quality assurance,  thus enhancing productivity and helping to digitise the process. The approach and apparatus took into account considerations like positional variance (placing the object in any alignment), various sizes of raw materials, lighting invariance, etc.

The approach taken by the Ignitarium Engineering team is detailed in subsequent sections:

  1. Dataset preparation
  2. Approach to solving the problem
  3. Labelling
  4. Training
  5. Accuracy
  6. Approach to using the application
  7. Application Screenshots
  8. Performance
  9. Conclusion

Dataset Preparation

In order to train a model, we must start with the appropriate amount of relevant data. For the footwear defect detection project, we were provided a variety of footwear raw materials samples of different types from our client’s production factories all over the country. A large dataset of images was generated by the team at Ignitarium by scanning the raw materials in different alignments. Here are some of the sample raw materials.

Approach to solving the problem

The approach to solving the problem involved using a combination of various image processing algorithms as well as Deep Learning detection algorithms using a special apparatus designed by Ignitarium.  Firstly, using a scanner we were able to retrieve a high resolution colour image of the raw material, then using complex image processing techniques, various preprocessing steps were carried out to ensure positional alignment.

For knowing more about positional alignment, the following links will be quite helpful:

  1. https://learnopencv.com/image-alignment-ecc-in-opencv-c-python/
  2. https://opencv-python tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_geometric_transformations/py_geometric_transformations.html

Deep learning was utilised for the automatic detection of cuts and holes in the raw materials. Semantic segmentation was used for the same and a number of architectures exist for the same like ParseNet, U-Net, etc. The following link will be helpful to gain more insights regarding the same.

U-Net consists of an encoder (downsampler) and decoder (upsampler). In order to learn robust features, and reduce the number of trainable parameters, a pretrained model can be used as the encoder. Thus, the encoder for this task can be a pretrained model like MobileNetV2, whose intermediate outputs will be used, and the decoder will be the upsample block.

Unet Model

[Source: https://raw.githubusercontent.com/zhixuhao/unet/master/img/u-net-architecture.png]

The following links are helpful for exploring about Unet:

  1. https://github.com/zhixuhao/unet
  2. https://www.tensorflow.org/tutorials/images/segmentation

TensorRT optimization

Inorder to speed up the inference results we had carried out TensorRT optimization on our models. TensorRT is an Nvidia software solution for generating optimized models for production deployment of Deep Learning Models. This is achieved by restructuring the graph to perform the operations much faster and more efficiently without changes to the underlying computation in the graph. Detailed information on TensorRT can be found here.

Labelling

Labelling is an important part of any ML-based system. Ignitarium used its own in-house labelling application for the annotation tasks involved in identification of cuts along the footwear raw materials, thus generating the corresponding mask images for the images scanned.

Training

Using the mask images obtained from the inhouse labelling application and the original scan images, training was carried out. Several pretrained encoder-decoder networks like resnet50_segnet, mobilenet_unet, resnet50_unet etc. are available that can be used based on the requirement.

Accuracy

Our custom semantic segmentation network was able to correctly identify the cuts in the raw materials with an accuracy of over 98%.

Approach to using the application

The application requires the user to store templates of each type of raw material they intend to test. It is a one time process for each template. The template creation process involves selecting the positions of holes and drawing the positions of cuts with a mouse pointer as well as marking specific regions that may require custom threshold as compared to other regions as well as entering other details regarding the template like size, article number, left or right orientation etc.

Application Snapshots

Here are some of the snapshots of the application:

This image shows the overall layout of the application

Application screenshot showing an example case of how the raw material tested is non defective. It has verified the position of the holes, cuts as well as border outlines in the template for any mismatch.

The screenshot here shows a feature of the application that allows the user to actually check the measures in S.I system (millimetre) between any points of interest in the raw material using a mouse pointer, thus eradicating the need to physically use a measuring tape or scale.

This particular view helps the user to get an idea regarding the regions that are either missing or in excess as compared to the template piece. The red boxes drawn indicate that the holes in the test piece are missing as compared to the original template. As the curves representing the cuts overlap it means that there are no variations in the position of the cuts.

Performance

Scanner’s processing time can vary with resolution used; for the specific scanner used, the total time taken for testing is around 35-40 seconds including scanning and generation of results. Normally the same process when carried out manually by a person, would be highly laborious requiring the verification of dimensions with instruments and visual inspection of raw piece with a template piece, that could take upto 3-4 minutes a piece.

Conclusion

Video explaining the features of the product is available in the following link:

Salient Features of the Footwear Defect Detection Application:

1) Identification of standard defects namely position of holes, cuts as well as outer surface shape of test pieces utilising Deep Learning and Image Processing techniques.

2) Mark custom regions in templates that are to be evaluated against custom tolerance values. Can  be utilized in cases where certain regions in the template are to be given more or less prominence.

3) Ability to use CAD files as well as Scanner Feed as Input.

4) Automatic Template Detection feature for identification of corresponding article, if previously stored, thus removing the need for manual selection by the user.

5) Search option for searching for templates based on article names.

6) Folder Structure for neatly organising Article Templates of different sizes

7) Re-orientation of test piece with respect to template either using shape matching or hole position of corresponding template.

8) Availability of different views after testing the Template to make the user understand visually the exact points of defects, if any.

9) Ability to Store and retrieve details from the Result Database regarding tests carried out with various filtering options.

10) Easy modification/ deletion of templates by privileged users.

11) Ability to assign custom privileges to different users of the application like Template Creation/ Modification, Create a new User and Test Analysis Overview.

12) Ability to specify custom tolerance values in the SI metric system for various dimensions taken into consideration for testing for each type of template.

13) Mandate the user to enter certain details like batch number and explanation if results generated by the application were overridden, all which will be stored in the database.

14) Export of report database details into Spreadsheet and PDF formats for Report generation. The details to be included can be controlled using filtering options

15) Ability to Import or Export Template Database.

Leave a Comment

Your email address will not be published. Required fields are marked *

one + 7 =

Scroll to Top

Human Pose Detection & Classification

Some Buildings in a city

Features:

  • Suitable for real time detection on edge devices
  • Detects human pose / key points and recognizes movement / behavior
  • Light weight deep learning models with good accuracy and performance

Target Markets:

  • Patient Monitoring in Hospitals
  • Surveillance
  • Sports/Exercise Pose Estimation
  • Retail Analytics

OCR / Pattern Recognition

Some Buildings in a city

Use cases :

  • Analog dial reading
  • Digital meter reading
  • Label recognition
  • Document OCR

Highlights :

  • Configurable for text or pattern recognition
  • Simultaneous Analog and Digital Dial reading
  • Lightweight implementation

Behavior Monitoring

Some Buildings in a city

Use cases :

  • Fall Detection
  • Social Distancing

Highlights :

  • Can define region of interest to monitor
  • Multi-subject monitoring
  • Multi-camera monitoring
  • Alarm triggers

Attire & PPE Detection

Some Buildings in a city

Use cases :

  • PPE Checks
  • Disallowed attire checks

Use cases :

  • Non-intrusive adherence checks
  • Customizable attire checks
  • Post-deployment trainable

 

Request for Video





    Real Time Color Detection​

    Use cases :

    • Machine vision applications such as color sorter or food defect detection

    Highlights :

    • Color detection algorithm with real time performance
    • Detects as close to human vison as possible including color shade discrimination
    • GPGPU based algorithm on NVIDIA CUDA and Snapdragon Adreno GPU
    • Extremely low latency (a few 10s of milliseconds) for detection
    • Portable onto different hardware platforms

    Missing Artifact Detection

    Use cases :

    • Detection of missing components during various stages of manufacturing of industrial parts
    • Examples include : missing nuts and bolts, missing ridges, missing grooves on plastic and metal blocks

    Highlights :

    • Custom neural network and algorithms to achieve high accuracy and inference speed
    • Single-pass detection of many categories of missing artifacts
    • In-field trainable neural networks with dynamic addition of new artifact categories
    • Implementation using low cost cameras and not expensive machine-vision cameras
    • Learning via the use of minimal training sets
    • Options to implement the neural network on GPU or CPU based systems

    Real Time Manufacturing Line Inspection

    Use cases :

    • Detection of defects on the surface of manufactured goods (metal, plastic, glass, food, etc.)
    • Can be integrated into the overall automated QA infrastructure on an assembly line.

    Highlights :

    • Custom neural network and algorithms to achieve high accuracy and inference speed
    • Use of consumer or industrial grade cameras
    • Requires only a few hundred images during the training phase
    • Supports incremental training of the neural network with data augmentation
    • Allows implementation on low cost GPU or CPU based platforms

    Ground Based Infrastructure analytics

    Some Buildings in a city

    Use cases :

    • Rail tracks (public transport, mining, etc.)
    • Highways
    • Tunnels

    Highlights :

    • Analysis of video and images from 2D & 3D RGB camera sensors
    • Multi sensor support (X-ray, thermal, radar, etc.)
    • Detection of anomalies in peripheral areas of core infrastructure (Ex: vegetation or stones near rail tracks)

    Aerial Analytics

    Use cases :

    • Rail track defect detection
    • Tower defect detection: Structural analysis of Power
      transmission towers
    • infrastructure mapping

    Highlights :

    • Defect detection from a distance
    • Non-intrusive
    • Automatic video capture with perfectly centered ROI
    • No manual intervention is required by a pilot for
      camera positioning

    SANJAY JAYAKUMAR

    Co-founder & CEO

     

    Founder and Managing director of Ignitarium, Sanjay has been responsible for defining Ignitarium’s core values, which encompass the organisation’s approach towards clients, partners, and all internal stakeholders, and in establishing an innovation and value-driven organisational culture.

     

    Prior to founding Ignitarium in 2012, Sanjay spent the initial 22 years of his career with the VLSI and Systems Business unit at Wipro Technologies. In his formative years, Sanjay worked in diverse engineering roles in Electronic hardware design, ASIC design, and custom library development. Sanjay later handled a flagship – multi-million dollar, 600-engineer strong – Semiconductor & Embedded account owning complete Delivery and Business responsibility.

     

    Sanjay graduated in Electronics and Communication Engineering from College of Engineering, Trivandrum, and has a Postgraduate degree in Microelectronics from BITS Pilani.

     

    Request Free Demo




      RAMESH EMANI Board Member

      RAMESH EMANI

      Board Member

      Ramesh was the Founder and CEO of Insta Health Solutions, a software products company focused on providing complete hospital and clinic management solutions for hospitals and clinics in India, the Middle East, Southeast Asia, and Africa. He raised Series A funds from Inventus Capital and then subsequently sold the company to Practo Technologies, India. Post-sale, he held the role of SVP and Head of the Insta BU for 4 years. He has now retired from full-time employment and is working as a consultant and board member.

       

      Prior to Insta, Ramesh had a 25-year-long career at Wipro Technologies where he was the President of the $1B Telecom and Product Engineering Solutions business heading a team of 19,000 people with a truly global operations footprint. Among his other key roles at Wipro, he was a member of Wipro's Corporate Executive Council and was Chief Technology Officer.

       

      Ramesh is also an Independent Board Member of eMIDs Technologies, a $100M IT services company focused on the healthcare vertical with market presence in the US and India.

       

      Ramesh holds an M-Tech in Computer Science from IIT-Kanpur.

      ​Manoj Thandassery

      VP – Sales & Business Development

      Manoj Thandassery is responsible for the India business at Ignitarium. He has over 20 years of leadership and business experience in various industries including the IT and Product Engineering industry. He has held various responsibilities including Geo head at Sasken China, Portfolio head at Wipro USA, and India & APAC Director of Sales at Emeritus. He has led large multi-country teams of up to 350 employees. Manoj was also an entrepreneur and has successfully launched and scaled, via multiple VC-led investment rounds, an Edtech business in the K12 space that was subsequently sold to a global Edtech giant.
      An XLRI alumnus, Manoj divides his time between Pune and Bangalore.

       

      MALAVIKA GARIMELLA​

      General Manager - Marketing

      A professional with a 14-year track record in technology marketing, Malavika heads marketing in Ignitarium. Responsible for all branding, positioning and promotional initiatives in the company, she has collaborated with technical and business teams to further strengthen Ignitarium's positioning as a key E R&D services player in the ecosystem.

      Prior to Ignitarium, Malavika has worked in with multiple global tech startups and IT consulting companies as a marketing consultant. Earlier, she headed marketing for the Semiconductor & Systems BU at Wipro Technologies and worked at IBM in their application software division.

      Malavika completed her MBA in Marketing from SCMHRD, Pune, and holds a B.E. degree in Telecommunications from RVCE, Bengaluru.

       

      PRADEEP KUMAR LAKSHMANAN

      VP - Operations

      Pradeep comes with an overall experience of 26 years across IT services and Academia. In his previous role at Virtusa, he played the role of Delivery Leader for the Middle East geography. He has handled complex delivery projects including the transition of large engagements, account management, and setting up new delivery centers.

      Pradeep graduated in Industrial Engineering and Management, went on to secure an MBA from CUSAT, and cleared UGN Net in Management. He also had teaching stints at his alma mater, CUSAT, and other management institutes like DCSMAT. A certified P3O (Portfolio, Program & Project Management) from the Office of Government Commerce, UK, Pradeep has been recognized for key contributions in the Management domain, at his previous organizations, Wipro & Virtusa.

      In his role as the Head of Operations at Ignitarium, Pradeep leads and manages operational functions such as Resource Management, Procurement, Facilities, IT Infrastructure, and Program Management office.

       

      SONA MATHEW Director – Human Resources

      SONA MATHEW

      AVP – Human Resources

      Sona heads Human Resource functions - Employee Engagement, HR Operations and Learning & Development – at Ignitarium. Her expertise include deep and broad experience in strategic people initiatives, performance management, talent transformation, talent acquisition, people engagement & compliance in the Information Technology & Services industry.

       

      Prior to Ignitarium, Sona has had held diverse HR responsibilities at Litmus7, Cognizant and Wipro.

       

      Sona graduated in Commerce from St. Xaviers College and did her MBA in HR from PSG College of Technology.

       

      ASHWIN RAMACHANDRAN

      Vice President - Sales

      As VP of Sales, Ashwin is responsible for Ignitarium’s go-to-market strategy, business, client relationships, and customer success in the Americas. He brings in over a couple of decades of experience, mainly in the product engineering space with customers from a wide spectrum of industries, especially in the Hi-Tech/semiconductor and telecom verticals.

       

      Ashwin has worked with the likes of Wipro, GlobalLogic, and Mastek, wherein unconventional and creative business models were used to bring in non-linear revenue. He has strategically diversified, de-risked, and grown his portfolios during his sales career.

       

      Ashwin strongly believes in the customer-first approach and works to add value and enhance the experiences of our customers.

       

      AZIF SALY Director – Sales

      AZIF SALY

      Vice President – Sales & Business Development

      Azif is responsible for go-to-market strategy, business development and sales at Ignitarium. Azif has over 14 years of cross-functional experience in the semiconductor product & service spaces and has held senior positions in global client management, strategic account management and business development. An IIM-K alumnus, he has been associated with Wipro, Nokia and Sankalp in the past.

       

      Azif handled key accounts and sales process initiatives at Sankalp Semiconductors. Azif has pursued entrepreneurial interests in the past and was associated with multiple start-ups in various executive roles. His start-up was successful in raising seed funds from Nokia, India. During his tenure at Nokia, he played a key role in driving product evangelism and customer success functions for the multimedia division.

       

      At Wipro, he was involved in customer engagement with global customers in APAC and US.

       

      RAJU KUNNATH Vice President – Enterprise & Mobility

      RAJU KUNNATH

      Distinguished Engineer – Digital

      At Ignitarium, Raju's charter is to architect world class Digital solutions at the confluence of Edge, Cloud and Analytics. Raju has over 25 years of experience in the field of Telecom, Mobility and Cloud. Prior to Ignitarium, he worked at Nokia India Pvt. Ltd. and Sasken Communication Technologies in various leadership positions and was responsible for the delivery of various developer platforms and products.

       

      Raju graduated in Electronics Engineering from Model Engineering College, Cochin and has an Executive Post Graduate Program (EPGP) in Strategy and Finance from IIM Kozhikode.

       

      PRADEEP SUKUMARAN Vice President – Business Strategy & Marketing

      PRADEEP SUKUMARAN

      Vice President - Software Engineering

      Pradeep heads the Software Engineering division, with a charter to build and grow a world-beating delivery team. He is responsible for all the software functions, which includes embedded & automotive software, multimedia, and AI & Digital services

      At Ignitarium, he was previously part of the sales and marketing team with a special focus on generating a sales pipeline for Vision Intelligence products and services, working with worldwide field sales & partner ecosystems in the U.S  Europe, and APAC.

      Prior to joining Ignitarium in 2017, Pradeep was Senior Solutions Architect at Open-Silicon, an ASIC design house. At Open-Silicon, where he spent a good five years, Pradeep was responsible for Front-end, FPGA, and embedded SW business development, marketing & technical sales and also drove the IoT R&D roadmap. Pradeep started his professional career in 2000 at Sasken, where he worked for 11 years, primarily as an embedded multimedia expert, and then went on to lead the Multimedia software IP team.

      Pradeep is a graduate in Electronics & Communication from RVCE, Bangalore.

       

      SUJEET SREENIVASAN Vice President – Embedded

      SUJEET SREENIVASAN

      Vice President – Automotive Technology

       

      Sujeet is responsible for driving innovation in Automotive software, identifying Automotive technology trends and advancements, evaluating their potential impact, and development of solutions to meet the needs of our Automotive customers.

      At Ignitarium, he was previously responsible for the growth and P&L of the Embedded Business unit focusing on Multimedia, Automotive, and Platform software.

      Prior to joining Ignitarium in 2016, Sujeet has had a career spanning more than 16 years at Wipro. During this stint, he has played diverse roles from Solution Architect to Presales Lead covering various domains. His technical expertise lies in the areas of Telecom, Embedded Systems, Wireless, Networking, SoC modeling, and Automotive. He has been honored as a Distinguished Member of the Technical Staff at Wipro and has multiple patents granted in the areas of Networking and IoT Security.

      Sujeet holds a degree in Computer Science from Government Engineering College, Thrissur.

       

      RAJIN RAVIMONY Distinguished Engineer

      RAJIN RAVIMONY

      Distinguished Engineer

       

      At Ignitarium, Rajin plays the role of Distinguished Engineer for complex SoCs and systems. He's an expert in ARM-based designs having architected more than a dozen SoCs and played hands-on design roles in several tens more. His core areas of specialization include security and functional safety architecture (IEC61508 and ISO26262) of automotive systems, RTL implementation of math intensive signal processing blocks as well as design of video processing and related multimedia blocks.

       

      Prior to Ignitarium, Rajin worked at Wipro Technologies for 14 years where he held roles of architect and consultant for several VLSI designs in the automotive and consumer domains.

       

      Rajin holds an MS in Micro-electronics from BITS Pilani.

       

      SIBY ABRAHAM Executive Vice President, Strategy

      SIBY ABRAHAM

      Executive Vice President, Strategy

       

      As EVP, of Strategy at Ignitarium, Siby anchors multiple functions spanning investor community relations, business growth, technology initiatives as well and operational excellence.

       

      Siby has over 31 years of experience in the semiconductor industry. In his last role at Wipro Technologies, he headed the Semiconductor Industry Practice Group where he was responsible for business growth and engineering delivery for all of Wipro’s semiconductor customers. Prior to that, he held a vast array of crucial roles at Wipro including Chief Technologist & Vice President, CTO Office, Global Delivery Head for Product Engineering Services, Business Head of Semiconductor & Consumer Electronics, and Head of Unified Competency Framework. He was instrumental in growing Wipro’s semiconductor business to over $100 million within 5 years and turning around its Consumer Electronics business in less than 2 years. In addition, he was the Engineering Manager for Enthink Inc., a semiconductor IP-focused subsidiary of Wipro. Prior to that, Siby was the Technical Lead for several of the most prestigious system engineering projects executed by Wipro R&D.

       

      Siby has held a host of deeply impactful positions, which included representing Wipro in various World Economic Forum working groups on Industrial IOT and as a member of IEEE’s IOT Steering Committee.

       

      He completed his MTech. in Electrical Engineering (Information and Control) from IIT, Kanpur and his BTech. from NIT, Calicut

       

      SUDIP NANDY

      Board Member

       

      An accomplished leader with over 40 years of experience, Sudip has helped build and grow companies in India, the US and the UK.

      He has held the post of Independent Director and Board Member for several organizations like Redington Limited, Excelra, Artison Agrotech, GeBBS Healthcare Solutions, Liquid Hub Inc. and ResultsCX.

      Most recently, Sudip was a Senior Advisor at ChrysCapital, a private equity firm where he has also been the Managing Director and Operating Partner for IT for the past 5 years. During his tenure, he has been Executive Chairman of California-headquartered Infogain Corporation and the non-Exec Chair on the board of a pioneering electric-2-wheeler company Ampere Vehicles, which is now a brand of Greaves Cotton Ltd.

      Earlier on in his career, Sudip has been the CEO and then Chairman India for Aricent. Prior to that, he had spent 25+ years in Wipro where he has been the Head of US business, Engineering R&D Services, and later the Head of EU Operations.

      Sudip is an active investor in several interesting startups in India and overseas, which mostly use technology for the social good, encompassing hyperlocal, healthcare, rural development, farmer support and e2W ecosystem. He also spends time as a coach and mentor for several CEOs in this role.

       

      SUJEETH JOSEPH Chief Product Officer

      SUJEETH JOSEPH

      Chief Technology Officer

       

      As CTO, Sujeeth is responsible for defining the technology roadmap, driving IP & solution development, and transitioning these technology components into practically deployable product engineering use cases.

       

      With a career spanning over 30+ years, Sujeeth Joseph is a semiconductor industry veteran in the SoC, System and Product architecture space. At SanDisk India, he was Director of Architecture for the USD $2B Removable Products Group. Simultaneously, he also headed the SanDisk India Patenting function, the Retail Competitive Analysis Group and drove academic research programs with premier Indian academic Institutes. Prior to SanDisk, he was Chief Architect of the Semiconductor & Systems BU (SnS) of Wipro Technologies. Over a 19-year career at Wipro, he has played hands-on and leadership roles across all phases of the ASIC and System design flow.

       

      He graduated in Electronics Engineering from Bombay University in 1991.

       

      SUJITH MATHEW IYPE Co-founder & CTO

      SUJITH MATHEW IYPE

      Co-founder & COO

       

      As Ignitarium's Co-founder and COO, Sujith is responsible for driving the operational efficiency and streamlining process across the organization. He is also responsible for the growth and P&L of the Semiconductor Business Unit.

       

      Apart from establishing a compelling story in VLSI, Sujith was responsible for Ignitarium's foray into nascent technology areas like AI, ML, Computer Vision, and IoT, nurturing them in our R&D Lab - "The Crucible".

       

      Prior to founding Ignitarium, Sujith played the role of a VLSI architect at Wipro Technologies for 13 years. In true hands-on mode, he has built ASICs and FPGAs for the Multimedia, Telecommunication, and Healthcare domains and has provided technical leadership for many flagship projects executed by Wipro.

       

      Sujith graduated from NIT - Calicut in the year 2000 in Electronics and Communications Engineering and thereafter he has successfully completed a one-year executive program in Business Management from IIM Calcutta.

       

      RAMESH SHANMUGHAM Co-founder & COO

      RAMESH SHANMUGHAM

      Co-founder & CRO

      As Co-founder and Chief Revenue Officer of Ignitarium, Ramesh has been responsible for global business and marketing as well as building trusted customer relationships upholding the company's core values.

      Ramesh has over 25 years of experience in the Semiconductor Industry covering all aspects of IC design. Prior to Ignitarium, Ramesh was a key member of the senior management team of the semiconductor division at Wipro Technologies. Ramesh has played key roles in Semiconductor Delivery and Pre-sales at a global level.

      Ramesh graduated in Electronics Engineering from Model Engineering College, Cochin, and has a Postgraduate degree in Microelectronics from BITS Pilani.