The Student Chapter of the International Association of Automation and Robotics in Construction at NYUAD (IAARC@NYUAD), in collaboration with CITIES and the S.M.A.R.T. Construction Research Group, organized the 2022 IAARC@NYUAD Image Labeling Challenge over the summer. The goal was to have an extensive dataset labeled based on the content of each image. A total of 40 students registered for this challenge. Students that labeled more than 10k images with few mistakes were ranked and given a price. The price consisted of an Amazon Gift Card provided by CITIES. Participating students were also given the possibility to get involved in ongoing research using the labeled image dataset.
The winners are:
| Name | Academic Level | Major |
| Tran Linh | Sophomore | Computer Science |
| Ronit Singh | Junior | Computer Science |
| Avishek Bist | Sophomore | Electrical Engineering |
| Vahagn Yeritsyan | Sophomore | Computer Science |
| Mushraf Ali Anver | Junior | Economics |
Each image in the dataset contained one or many of the 13 labels or categories representing different categories such as HVAC ducts, scaffolding, formwork, construction workers, or rebar. Students used the SMART Swipe App to classify each image according to the different categories or labels. At the end of the challenge, a total of 103,021 classifications were made. The following was achieved for each category.
| Category (Label) | Status | Category(label) | Status |
| HVAC rectangular ducts | 82% | Formwork | 43% |
| Circular ducts, pipes, and conduits | 33% | Rebar(steel) | 54% |
| Walls | 68% | Workers/ People | 100% |
| Studs (metal, wood) | 100% | Ceiling support systems | 67% |
| Drywall panel | 62% | Electrical items, wires | 38% |
| Drywall insulation filler | 100% | Labels/signs/barricade cones/tapes | 57% |
| Scaffolding | 79% |
A sample of the labeled data that resulted from the challenge can be found here.
The result of this challenge is a big push toward developing a multi-label classifier that will be followed by object localization and segmentation. The current status of the data labeling done so far can be a significant asset to conducting research in automating worker detection and safety monitoring in construction sites or construction status and inspection of drywalls and HVAC elements. Similarly, given the size of the dataset, there is the possibility to use that for research in areas of construction site space optimization, automated safety policy compliance, and other related topics.
About the SMART Swipe App
The app used in this challenge is a cross-platform mobile and web application developed by Pi Ko, an undergraduate research assistant under the supervision of Prof. Borja Garcia de Soto from the S.M.A.R.T. Construction Research Group at New York University Abu Dhabi. Usually, the generation of classified image data is labor intensive, slow, and not engaging – the task involves the users sitting in front of computers and clicking the correct labels for the images.
SMART Swipe aims to make this process fun, quick and engaging by making the image labeling process a swiping process, where the user can swipe left and right to say if the label is true or not on any mobile device, wherever they are. The swiping process takes only two seconds for each image and is more interactive. In terms of technology, the application is powered by modern industrial web technologies – ReactJS for front-end development, NodeJS for backend development, and MongoDB for fast and efficient database management. The application also has verification functions for administrators to overwrite any falsely labeled data, visualizations of project completion progress, and control images to determine if the data is labeled correctly. Currently, SMART Swipe hosts 11,639 images and 13 categories/labels but can be expanded and customized as needed.
A demonstration of the SMART Swipe App can be seen in this video.