Blind and Visually Impaired Community
↑
1
↓
Engineering project: wearable device to detect head level obstacles (Is audio feedback a terrible idea?) (self.Blind)
submitted by cyrusyoman
Hi everyone, I'm an engineering student at Simon Fraser University, and I wanted some feedback on a device I am building for the visually impaired and the blind. My group has began creating a wearable device for a course which will be hidden in a hat that will be used to detect head level obstacles, and communicate these objects to the user. This device is designed to be used with other forms of safe navigation, such as white canes or guide dogs.
There are two parts to the system, one main part which detects physical head level obstacles, and vibrates motors to communicate how far these objects are. The other secondary part will analyze live video filmed ahead of the user which will detect relevant street signs or crosswalks. This second part is a completely optional add on to the main headpiece.
We are running into an issue here. For the head-level obstacles, we are using vibrational motors to tell the user where the object is, but were unsure how to communicate to the user what signs lay ahead. We thought of using audio feedback, but many people think that may be overwhelming to the user.
This is where I ask you, do you think that audio feedback is a bad idea? Do you have other recommendations? This part is optional, so audio feedback isn't necessary. Is audio feedback a terrible idea?
​
TL;DR: Designing a product, wondering if audio feedback is a bad idea for the visually impaired and the blind
There are two parts to the system, one main part which detects physical head level obstacles, and vibrates motors to communicate how far these objects are. The other secondary part will analyze live video filmed ahead of the user which will detect relevant street signs or crosswalks. This second part is a completely optional add on to the main headpiece.
We are running into an issue here. For the head-level obstacles, we are using vibrational motors to tell the user where the object is, but were unsure how to communicate to the user what signs lay ahead. We thought of using audio feedback, but many people think that may be overwhelming to the user.
This is where I ask you, do you think that audio feedback is a bad idea? Do you have other recommendations? This part is optional, so audio feedback isn't necessary. Is audio feedback a terrible idea?
​
TL;DR: Designing a product, wondering if audio feedback is a bad idea for the visually impaired and the blind
KillerLag 4 points 3y ago
Audio feedback isn't necessarily bad, but it can get overwhelming. If the audio feedback is very seldom, then it works as an alert to something hazardous. But if it keeps going off constantly, then that becomes becomes a greater problem, because it could block off other sounds.
It sounds like this device activate for something that is hanging at head level (such as a sign or a tree branch), but would it activate if you were walking towards a wall (or a pole)? So something that is at head level, but continues down to the ground. Because those are much more common, and if it goes off constantly because of that (say, you are walking in a hallway, or near a building), then the audio would become more of a hindrance. Especially if someone is trying to cross the street, so they are listening for traffic, but there are light posts and other poles all around.
It sounds like this device activate for something that is hanging at head level (such as a sign or a tree branch), but would it activate if you were walking towards a wall (or a pole)? So something that is at head level, but continues down to the ground. Because those are much more common, and if it goes off constantly because of that (say, you are walking in a hallway, or near a building), then the audio would become more of a hindrance. Especially if someone is trying to cross the street, so they are listening for traffic, but there are light posts and other poles all around.
cyrusyoman [OP] 1 points 3y ago
This is awesome feedback! If something like a pole, or a wall are beside you, the motors will buzz, but there will be no audio. Is this an issue? Would a speaker work better than a headset? We are definitely going to try to limit the amount of audio that gets fed to only crucial information.
KillerLag 3 points 3y ago
Something that covers the ears would block out a significant portion of sound, so when I discuss listening to things with clients (like GPS directions and such), I usually suggest bone conduction headphones (like AfterShokz) or really directed audio (liked Bose Frames, but those are pretty pricey).
Vibrations do work, but they aren't always the most comfortable on the face. I vaguely recall there was a set of glasses with an ultrasonic sensor and a vibration motor in the temples a few decades ago (I can't recall the name), and the vibrations were not pleasant. That may have just been me, though (even AfterShokz's vibrations feel uncomfortable for me on high volumes).
Vibrations do work, but they aren't always the most comfortable on the face. I vaguely recall there was a set of glasses with an ultrasonic sensor and a vibration motor in the temples a few decades ago (I can't recall the name), and the vibrations were not pleasant. That may have just been me, though (even AfterShokz's vibrations feel uncomfortable for me on high volumes).
Cptn_dropbear 2 points 3y ago
Not sure of you have heard or know about prof Leslie Kay's work with sonic glasses and hand held torch. He has done a lot of publications and information about his devices dating back to the 1970's in England and New Zealand
A lot of early devices used audio on a musical scale and hard and soft tones to identify if object was stationary, moving towards or away and the last version before he died could tell of objects were metal, wood or concrete. Giving audio tones to identify.
Here is a link to an interview with him
https://youtu.be/zpifkY2C5og
I hope it helps with your research and development best of luck.
I can also put you in contact with one of the original testers and knew him very well through the years and was part of his research team.
A lot of early devices used audio on a musical scale and hard and soft tones to identify if object was stationary, moving towards or away and the last version before he died could tell of objects were metal, wood or concrete. Giving audio tones to identify.
Here is a link to an interview with him
https://youtu.be/zpifkY2C5og
I hope it helps with your research and development best of luck.
I can also put you in contact with one of the original testers and knew him very well through the years and was part of his research team.
cyrusyoman [OP] 1 points 3y ago
>Thank you so much for the response! I will definitely take a look :)
Rethunker 1 points 3y ago
Lots and lots of research has been done on this very topic for decades. There are already many apps and devices with audio feedback and haptic feedback. If you ask everyone in your group to research a different slice of what's already on the market--apps, ultrasound devices, smart glasses, smart canes (ugh), etc.--you'll find out a lot in a week.
Grab a copy of Foundations of Orientation & Mobility, look up the references, and keep going! It's good to try to solve a problem with your group just to see what you devise first, but don't get too far before you read about what's been done so you don't end up reinventing the wheel. Whole careers have been spent creating and testing navigation and safety systems, and you'll benefit a lot from reading papers by it.
Sensor substitution has been around for half a century. Having newer tech doesn't solve some of the fundamental problems. Read and find out what those problems are.
Some other resources are posted here:
$1
Vibrating motors for someone in motion is something you're right to think could be problematic. For audio . . .well, talk to some blind people locally and you'll quickly figure out there are a few important considerations.
Think very carefully about legal liability associated with detecting street signs and crosswalks. **Consult with a lawyer who specializes in liability.** You're not the first to try outdoor recognition tasks, and there are good reasons why assistive tech companies with engineering teams and large budgets have shied away from it. This is not simply a recognition task (which, incidentally, is even harder than it seems). Consider what happens if your hardware fails at an inopportune moment, or even whether your software development approach can guarantee code that is safe.
I realize this may be a student project, but you mentioned designing a "product," and you could end up spending a lot of effort only to discover some troubling news later. Best to figure out some of the road blocks now.
For sidewalks and other outdoor recognition tasks you should see some initial success, but then you'll run into some very serious problems. Ask yourself what the best recognition accuracy is for **any** deep learning task. Consider whether this is sufficient for safety. (It isn't.)
A lot of assistive tech builds in serious flaws from the start that can't (or at least likely won't) be overcome later. With some research you could likely leapfrog other student projects.
Grab a copy of Foundations of Orientation & Mobility, look up the references, and keep going! It's good to try to solve a problem with your group just to see what you devise first, but don't get too far before you read about what's been done so you don't end up reinventing the wheel. Whole careers have been spent creating and testing navigation and safety systems, and you'll benefit a lot from reading papers by it.
Sensor substitution has been around for half a century. Having newer tech doesn't solve some of the fundamental problems. Read and find out what those problems are.
Some other resources are posted here:
$1
Vibrating motors for someone in motion is something you're right to think could be problematic. For audio . . .well, talk to some blind people locally and you'll quickly figure out there are a few important considerations.
Think very carefully about legal liability associated with detecting street signs and crosswalks. **Consult with a lawyer who specializes in liability.** You're not the first to try outdoor recognition tasks, and there are good reasons why assistive tech companies with engineering teams and large budgets have shied away from it. This is not simply a recognition task (which, incidentally, is even harder than it seems). Consider what happens if your hardware fails at an inopportune moment, or even whether your software development approach can guarantee code that is safe.
I realize this may be a student project, but you mentioned designing a "product," and you could end up spending a lot of effort only to discover some troubling news later. Best to figure out some of the road blocks now.
For sidewalks and other outdoor recognition tasks you should see some initial success, but then you'll run into some very serious problems. Ask yourself what the best recognition accuracy is for **any** deep learning task. Consider whether this is sufficient for safety. (It isn't.)
A lot of assistive tech builds in serious flaws from the start that can't (or at least likely won't) be overcome later. With some research you could likely leapfrog other student projects.
Prefect316 1 points 3y ago
How is the device interpreting the video it records? Are you using some kind of AI or something?
It sounds like you're attempting to reinvent a wheel of sorts with this. As a totally blind person who is also quite tall, I bump my head on many things. I don't imagine wanting to pay much just to not hit my head.
However, if this video is actually interpreted correctly by the device to the point where it can read street signs and identify sidewalks, you might have something blind folks might buy. I just think right now there's no demand for what you're developing, and I don't know for sure, but cannot imagine AI is at a point where it can reliably work like I wondered in my first paragraph.
It sounds like you're attempting to reinvent a wheel of sorts with this. As a totally blind person who is also quite tall, I bump my head on many things. I don't imagine wanting to pay much just to not hit my head.
However, if this video is actually interpreted correctly by the device to the point where it can read street signs and identify sidewalks, you might have something blind folks might buy. I just think right now there's no demand for what you're developing, and I don't know for sure, but cannot imagine AI is at a point where it can reliably work like I wondered in my first paragraph.
cyrusyoman [OP] 1 points 3y ago
Hi! Yes, we're using machine learning to detect these signs and crosswalks. You would be surprised, machine learning is also what powers the self driving feature of the tesla, which does almost the same thing! As I said, we're nowhere near a production phase, this is more of a design project, but I appreciate the feedback!
RJHand 1 points 3y ago
Not a bad idea but I personally wouldn't use it, I got a cane for a reason lol tbh.
This nonprofit website is run by volunteers.
Please contribute if you can. Thank you!
Our mission is to provide everyone with access to large-
scale community websites for the good of humanity.
Without ads, without tracking, without greed.
scale community websites for the good of humanity.
Without ads, without tracking, without greed.
©2023 HumbleCat Inc • HumbleCat is a 501(c)3 nonprofit based in Michigan, USA.