Bring your karma
Join the waitlist today
HUMBLECAT.ORG

Blind and Visually Impaired Community

Full History - 2022 - 07 - 09 - ID#vvd0by
4
"Look ma, no electrodes!" - No retinal implant, no brain implant, just your ears and your smartphone (self.Blind)
submitted by seeingwithsound
The vOICe web app $1 OIC? Oh, I see! Give it a try with your iPhone or Android smartphone. What does your white cane sound like? Right, a tone with rising or falling pitch depending on its orientation in the camera view of your smartphone. Have fun exploring sensory substitution for vision! BTW, do not get frustrated with the complexity of hearing out real-life imagery.

Background information and tutorial at $1
mehgcap 5 points 1y ago
I forgot the web app works in Safari on iOS these days. My main reason for not using this is really just convenience. If I could magically have the camera see my point of view, I'd probably use it a lot more. I'm aware of Android glasses and the like, but I don't believe there are cheap, sleek options out there yet. This isn't specific to The vOICe, either, it's the same with those AI image recognition apps, remote help apps, and similar. I just don't have an easy way to let my phone see what it needs to see when I'm actually out somewhere these solutions would help, and I don't want to buy, charge, and wear heavy camera glasses.

I have long thought that The vOICe has a lot of potential. With some practice, one can get pretty good with it, especially for larger objects. Work on vibration belts and other augmentation is pretty interesting, even if I don't know that I'd ever use such things personally. There's just no way I know of to use this app conveniently, which is no fault of the creator.
seeingwithsound [OP] 1 points 1y ago
It is true, the wait is for good hassle-free and affordable smart glasses, glasses that I could recommend without hesitation. That in turn depends on having a mass market killer app for the sighted consumer, because the current consumer use cases are not compelling enough for sighted people to want to wear smart glasses throughout the day. Sometimes I recommend as an interim solution to use a virtual reality headset that you can put a smartphone in: sure, it's large and heavy, with a silly appearance, but it is affordable for most blind people, smartphones are mature and have good cameras, while many blind people already have at least one smartphone, and appearance need not be a big issue at home or at work. Definitely not convenient, but it allows one to build up experience in interpreting the soundscapes, which is not something that one can master overnight once convenient smart glasses, say from Apple, hit the market. The vOICe web app is as platform-agnostic as can be, running not only on the iPhone (in the Safari browser) and on Android smartphones, but also on Google Glass, Vuzix Blade, and other Android smart glasses, and even on for instance MagicLeap One glasses (Helio browser), and probably Microsoft HoloLens (Edge browser, untested though). So chances are that The vOICe web app will run on Apple glasses right from day one, provided that Apple does not leave out common browser functionality (for instance, Facebook blocks pixel-level camera access on their Oculus Quest 2 headset, or else The vOICe web app would work on the Quest 2). So for now one cannot escape making some difficult trade-offs.
ukifrit 3 points 1y ago
I don't have the tools or even the time to try this out. But it seems like you could have some fun with it. Brilliant stuff!
seeingwithsound [OP] 1 points 1y ago
Thank you! Playing around with it is always a good start, such that things start brewing in the back of your mind, but yes, your time is a precious resource.
ukifrit 2 points 1y ago
I don't think it would be that practical for us common human beings without head mounted cameras and stuff, TBH. But I see it as a nice mechanic for accessible games, I'm sure people can think of other ways to use it too.
seeingwithsound [OP] 2 points 1y ago
I fully agree that a head-mounted (or at least chest-mounted) camera setup is the way to go, for consistent sensory feedback on the camera view in relation to your body posture and (head) movements. As mentioned before, a VR headset holding your smartphone is an affordable way to get started and learn what The vOICe can mean for you in immersive use, while preventing that you waste much money on something that may turn out not to suit your needs and interests. I have no desire to talk anyone into spending money that they will later regret spending. The biggest investment is the user's time, because learning to see with sound is not for the faint of heart, and it is hard to predict if benefit will outweigh the training effort and time. For some, even a semblance of sight can mean a lot even if it offers negligible functional vision, while for others it is functional vision that matters while they couldn't care less about how vision-like the experience is. People differ, and so do their use cases and interests.
ukifrit 2 points 1y ago
I have zero nostalgia of my previous sight, as I was always legally blind, with only collour and light sensibility. But I like this kind of stuff E.G, for quicker object detection, as the sound clues are way faster than a voice saying what and where stuff is. I'll give it a try in my vacations, just for fun.
SiriuslyGranger 2 points 1y ago
This is fascinating indeed, and very cool, but not sure ultimately if I am that interested. Seeing by feel or proprioceptions is good enough for me.
seeingwithsound [OP] 1 points 1y ago
Thank you. Perhaps it is something for those who have an uncontrollable yearning for a semblance of sight, but there can be plenty of other more prosaic reasons; I do not know the answers, but my guess is that it is good to have at least the options available.
SiriuslyGranger 2 points 1y ago
Yeah, I can agree with this probably good for someone like that. I don’t realy. But the more options out in the market the better I agree.
blind_cowboy 2 points 1y ago
I would be interested in trying this while target shooting with a pistol. I can already aim where my head points.
seeingwithsound [OP] 1 points 1y ago
You probably need some sort of telescope that you can connect a webcam to, plus adequate visual contrast for the target, but perhaps you have other ways of aiming.
Aggressive-Yoghurt31 2 points 1y ago
do we know anyone who has mastered this and uses it,? I want to hear from them if they exist
seeingwithsound [OP] 2 points 1y ago
Good question! Vision is extremely multi-faceted, making your question difficult to answer. I do not know of anyone yet who fully masters it to the level allowed by the attainable visual acuity, say in the low vision 20/400 range, in the sense of recognizing arbitrary objects in an unfamiliar environment. However, one of the Russian users of smart glasses running The vOICe could efficiently locate, reach for and fetch multiple bright objects cast like dice on a table top, or follow a painted line while running on a sports track. This same user also beat the only two Argus II retinal implant recipients in Russia, at the Neurothlon 2018 blind navigation and object identification contest, and he won by a large margin. Note that the implants cost $150,000 versus $200 for the smart glasses running The vOICe. So the limited evidence that we have strongly suggests that The vOICe outperforms state-of-the-art retinal implants and brain implants (e.g. the Orion I from Second Sight) in terms of functional vision, but we still do not know the limits of what is possible through extensive practice with adequate training. I do not have accessible video material about this Russian user, but you can ask a sighted friend to check out and describe to you the videos on Twitter at $1 and $1

You may also find some video clips with Jens Naumann interesting: he is one of the 16 former recipients of a Dobelle brain implant for vision, making the $1 in 2002, and although he is not a really proficient user of The vOICe due to his wide range of interests, he does use The vOICe, understands the system very well, and demonstrates and discusses it for instance in YouTube videos at https://youtu.be/pyoIq8euEGs and https://youtu.be/0GsDfWqCMqw
[deleted] 2 points 1y ago
[removed]
seeingwithsound [OP] 1 points 1y ago
Yes, $1 can do that too, as is explained on the web page $1. Of course it can be very hard to interpret with all the fast-paced visual clutter, but in otherwise inaccessible 3D games even the tiniest hints can be helpful. Sometimes a combination with color filters helps. Hope it works for you!
Ok_Scar9881 2 points 1y ago
I'm trying to get a good setup going for my PC. I'm wondering how webCam glasses would do? For now, we've devised a way to attach a webCam to a pair of glasses, and use earbuds. I know, it's probably going to look weird, but since I'm going to be using this at home, the looks won't matter. The webCam is one that has a wider view than webCam glasses we've looked at.
seeingwithsound [OP] 1 points 1y ago
Just for my understanding: are you thinking of putting a notebook PC running Microsoft Windows in a backpack of sorts, with USB camera glasses wired to the USB port of the notebook PC? That will work, and again there are multiple options: you can run The vOICe web app as mentioned in my initial post, in a browser such as Chrome, Firefox or Edge. However, there also exists a native version of The vOICe for Windows, available from $1, so for the discussion it is good to know if you plan to use $1 or $1.

Next, there exist various USB camera glasses, available from vendor sites such as eBay or Amazon, but in my experience the device specifications often do not mention if the glasses can act as a UVC compliant webcam even if they can, because their primary intended use is as an independent video recording device that stores video on an SD card for later transfer to a PC, and that is of course not the functionality that we are interested in: we need to stream and process live video. Therefore some degree of trial and error is often unavoidable with ready-made camera glasses. Your improvised camera glasses with a webcam attached to glasses is your safest bet, but for camera glasses with a better appearance also check out my web page on $1. Note that this page has not been updated for several years now since Microsoft lost the battle for mobile, so some of the information there is likely outdated and some links may be broken, but it still describes various details that can be useful to know.
[deleted] 2 points 1y ago
[deleted]
seeingwithsound [OP] 2 points 1y ago
Thank you. Yes, it is possible to assemble affordable smart glasses based on the Raspberry Pi, and several blind people have done work in that direction. Note that with Raspberry Pi one has the option to run The vOICe web app in a browser, as described above, but it is also possible to run a native Android version of The vOICe in an Android port for Raspberry Pi. For instance, Jacob Kruger from South Africa made an accessible $1, while Pranav Lal from India made a $1.

However, beyond tech-savvy blind people creating smart glasses for their personal use or on a very small scale, how does one organize global availability with sales, distribution, servicing etc? Until there is a commercially viable market for smart glasses for blind people, scalability remains a major issue. Furthermore, Raspberry Pi boards are not yet small enough to create smart glasses with a sleek and elegant appearance. Some blind people therefore instead buy the smallest Android smartphone that they can find, and use the extra phone for the sole purpose of running The vOICe for Android, with the mini-phone mounted in improvised glasses-like frames. Also not sleek, but technically much less demanding to assemble, and in a comparable price range as a Pi based setup. In any case, despite all the challenges, the general direction that you sketch is very interesting and worthwhile, so one needs to keep pushing... BTW, scalability is also a major challenge in organizing training, because learning to see with sound is really hard and not that many people can make it work without some level of 1-on-1 training support.
seeingwithsound [OP] 1 points 1y ago
>it could be pretty cheap to compile the needed parts and 3d print a mount to hold it all together.

Related to the hardware tinkering idea for smart glasses is also a long discussion thread of 9 years ago on the Raspberry Pi forum, titled "$1". At that time The vOICe web app did not exist yet.
[deleted] 2 points 1y ago
[deleted]
[deleted] 1 points 1y ago
[deleted]
[deleted] 1 points 1y ago
[deleted]
seeingwithsound [OP] 0 points 1y ago
Next put a bright-colored coffee cup on a dark table top (a dark cloth will do), and try to locate and grab the cup based on the soundscapes that you hear when pointing the camera of your phone in the general direction of the cup. Ideally you would for this exercise be wearing your smartphone in a virtual reality headset, or have your smartphone chest-mounted, because having your "eye" in your hand is perceptually very confusing due to all the degrees of freedom of your joints, but hopefully you will still get some idea of what this is all about. No more groping or sweeping once mastered.
[deleted] -1 points 1y ago
[deleted]
seeingwithsound [OP] 1 points 1y ago
Maybe you can rephrase your question, because it reads like you are looking for drugs.
[deleted] -1 points 1y ago
[deleted]
seeingwithsound [OP] 2 points 1y ago
Why these toxic comments? I'm open to criticism, but not to trolling.
This nonprofit website is run by volunteers.
Please contribute if you can. Thank you!
Our mission is to provide everyone with access to large-
scale community websites for the good of humanity.
Without ads, without tracking, without greed.
©2023 HumbleCat Inc   •   HumbleCat is a 501(c)3 nonprofit based in Michigan, USA.