The Street Scanner team and their namesake robot.
From top to bottom, left to right: Gwee Yong Ta (Information Systems Technology and Design), Wong Zhi Cong (Engineering Product Development), Tan Joon Kang (Information Systems Technology and Design), Jerome Heng (Information Systems Technology and Design), Yai Yan Lin (Information Systems Technology and Design), Joel Ong (Information Systems Technology and Design), and Lui Yan Le (Engineering Product Development).
The distinctive call of the koels, the rhythmic brushing of straw brooms on the pavement, and the occasional dinging of bicycle bells—yes, these are the familiar sounds of Singapore’s streets that we’ve come to know and love. But throw in a “beep boop” here and there, and you’ll get the sounds of these very streets, albiet in the near future, when autonomous robots serving various purposes are rolled out onto our neighbourhoods.
Now, this isn’t an entirely foreign concept—in fact, these days it’s easy to spot robot servers in restaurants and eateries, cleaning robots in malls, and even room service robots in hotels. So if having robots roving and cleaning the streets shouldn’t come as a surprise to us, then perhaps the right question to ask here is—why?
One For The Team
Besides the usual fallen leaves and broken tree branches, other things commonly found on our streets include cracked pavements, and man-made rubbish such as plastic bottles, used tissue, face masks, and so on. As much as manual labourers can and are deployed to clean these pavements, there are a lot of hidden processes in place and government bodies involved in the overall maintenance of our streets.
For instance, for a given stretch of pavement, the bicycle path could be managed by the Land Transport Authority (LTA), the park connector by NParks, and the grass patch by National Environment Agency (NEA). Currently, during manual inspections, a significant amount of back-and-forth communication is required between these governing bodies to determine the necessary flow of manpower and action.
But, by digitising this tedious process via a robot that can take note and keep track of what needs to be done and where, information can be relayed to the relevant parties more efficiently, lowering manpower costs.
“It’s not very easy to get a robot to repair cracks in the pavement,” Joel explains. “But what is easy to do is to send a robot down to tell you where you need to direct your manpower, and what to prioritise—that’s what we’re trying to solve.”
Additionally, the team shares that through their preliminary interviews with private contractors and government organisations, other big issues that the industry is facing are a lack of locals who are willing to do such manual jobs, and the limitations of the foreign worker quota.
“Basically, we’ve designed a system that provides the data required for these organisations to more efficiently allocate and manage the manpower that they have,” Joon Kang shares.
In collaboration with their industry partner, Continental Automotive Singapore, the team fitted their system onto Continental’s last mile delivery robot Corriere, and used a combination of sensors, cameras and Artificial Intelligence (AI) to allow the resulting robot to detect cracks and small debris on pavements. Essentially, they created an autonomous robot that can fully automate the pavement inspection process.
With an AI model trained on a total of 8000 images from datasets like Trash Annotations in Context (TACO) and various street pavement datasets, Street Scanner is able to detect pavement cracks and common litter with high accuracy, and uploads real-time updates onto a fully-equipped web dashboard.
A Bumpy Ride
As stable as the ideation and development process sounds, the team shares that their journey wasn’t quite as smooth-sailing as it seems. Joel tells us that one of the struggles the team faced early on was simply finding people to verify the team’s assumptions about the problem at hand.
“As complete outsiders of the system of inspection and pavement maintenance, it was difficult to find people who were willing to talk about their issues with us,” Joel says. “It was a lot about us asking people “Do you know this?”, and if they didn’t, then we would ask “Do you know a person who knows this?” and ad infinitum.”
Eventually, the team resolved the issue through sheer perseverance, but were then presented with another challenge—limited access to the Corriere robot. The robot was housed in Continental Automotive Singapore’s headquarters in Boon Keng, while the team was based in SUTD in the far east side of the island. This meant that Zhi Cong and Yan Le, the two Engineering Product Development students on the team, had to spend time traveling back and forth to take down the robot’s measurements, come up with a computer model, fit it onto the robot, and repeat the entire process whenever anything didn’t fit.
Through these challenges and the initial ideation and brainstorming process, the team tells us that they couldn’t have gotten through it all without each other. Made up of two Engineering Product Development and five Information Systems Technology and Design students, including Joel, who’s taking a double degree in Business, the team attributes the success of their project to the multidisciplinary nature of the team.
“It’s really been an eye-opener being able to work so closely with students from the other pillars, who have very different perspectives on how to tackle the problem statement that we received, as well as how to develop solutions and solve problems,” Yan Le says.
By exploring multiple possible avenues and solutions at every stage of their project, everyone in the team has been able to contribute their expertise, whether it be from their coursework, internships, or external experiences, to come up with a cohesive solution that plays well to each of their strengths.
So when the team overcame all odds and finalised their fully-functioning robot right before the Capstone showcase, Joel tells us that the team was full of excitement, and even started screaming with joy and relief—most of which they managed to catch on camera. (And no, they’re not willing to share those clips, unfortunately!).
Yan Lin showing off the team’s robot to visitors during the 2022 Capstone Showcase, which was held physically in SUTD and open to the public for the first time since 2019.
People First
Excitement aside, one of the questions that inevitably comes up with the automation of traditionally tedious tasks is the role of manual labour in a future powered by technology and robotics. Now that we have innovations like Street Scanner, what does the future of manual labour look like?
Yan Le explains that for him, a future with robotics is more about automating redundant tasks, and exploring how robots can work alongside humans, as opposed to having robots replace manual labour per se. A good example he quotes is the use of automated book shelving robots at the Tampines Regional Library. With these robots, librarians are thus free to do higher value work such as research.
“Likewise, we see Street Scanner as a solution to reducing the manpower required in the maintenance of our pavements, so workers don’t necessarily have to work such long hours,” Yan Le says.
On the other hand, Joel approaches this question from an economics standpoint. He explains that people are employable if their productivity output is higher than the amount that they’re paid to work, and robots have the potential to help people be more productive in the same amount of time.
“Ideally, like with other forms of technology, robotics is not intended to replace humans, but to make them more prosperous and productive, and to be used for overall social good,” Joel says. Unfortunately, he adds that in this current moment, things might be different, because as long as the cost of manual labour is lower than the cost of developing a robot for a specific usage, employers will continue to opt for manual labour.
But of course, with teams like these continually innovating and pushing the limits of what robots can do, we can most certainly expect robots to become increasingly useful in the future.
The prototype of Street Scanner that the team developed for their Capstone is remote-operated, but can be further modified to be fully autonomous.
“The robotics team at Continental, whom we have been working with, gave us positive feedback on our project’s progress,” Yan Le tells us.
He adds that the team will be handing the project off to Continental after graduation, but that it’s been a fun ride for himself and his teammates. “This is why I enjoy doing projects so much—the moments of adrenaline and exhilaration when everything falls into place and works are always fun and memorable.”
For more information about what Street Scanner can do, visit https://capstone2022.sutd.edu.sg/projects/continental-sensor-system
