Blind and Visually Impaired Community
↑
13
↓
Made an iPhone app that can describe your surroundings and photos (self.Blind)
submitted by taleser
Hello r/Blind !
I'm a high school student and I've spent the last few months developing Perigo Sight, which is an app that uses artificial intelligence to speak out contextual descriptions of your photos. On newer phones, it runs quite quickly, but will occasionally give inaccurate descriptions. If you're curious, you can get more information at $1 or download it at $1
This is a project I really care about and I'm open to any feedback, questions, or feature requests you have about Perigo Sight. There is also an app called Aipoly that I highly recommend for more basic object classification.
EDIT: By a remarkable coincidence, Perigo was launched on the same day as Microsoft's Seeing AI app. At a glance it looks extremely useful, and has a beta feature that works similarly to Perigo. I hope you can get to try both of them out!
EDIT 2: Okay, so I got some feedback from a user, and clearly it is not nearly accessible enough right now. Working on a fix, update coming tomorrow.
I'm a high school student and I've spent the last few months developing Perigo Sight, which is an app that uses artificial intelligence to speak out contextual descriptions of your photos. On newer phones, it runs quite quickly, but will occasionally give inaccurate descriptions. If you're curious, you can get more information at $1 or download it at $1
This is a project I really care about and I'm open to any feedback, questions, or feature requests you have about Perigo Sight. There is also an app called Aipoly that I highly recommend for more basic object classification.
EDIT: By a remarkable coincidence, Perigo was launched on the same day as Microsoft's Seeing AI app. At a glance it looks extremely useful, and has a beta feature that works similarly to Perigo. I hope you can get to try both of them out!
EDIT 2: Okay, so I got some feedback from a user, and clearly it is not nearly accessible enough right now. Working on a fix, update coming tomorrow.
fastfinge 2 points 6y ago
Wow, amazing that you were able to do this while still in school! Unfortunately, you kind of picked terrible timing for the release; Microsoft's Seeing AI app is all over the press/blogs right now. So you're going to have a bit of a struggle getting any coverage. However, seeing AI does require an Internet connection, and if you read the privacy policy, it's kind of terrifying.
A few questions:
1. Have you heard of, or tried, $1? I assume you're using a similar approach to them. How do you feel your app compares? I can (and will at some point) do a real-world test comparing them, but it's always nice to here the developer's thoughts.
2. Can your app do OCR? I didn't see that mentioned anywhere. But it's really nice to have both functions in a single app. Especially if I'm using it on photos; most photos today have captions, most products have writing on them, etc. So it's a big win to be able to get both functions in a single app, rather than having to take the same picture with two different apps. Right now, the only apps that offer both functions in one app are $1 (not free), and Microsoft's seeing AI app (not private).
I'm getting to a place where I feel like I need to have four different image identification apps installed, and it's kind of frustrating. One does better OCR, one can scan bar codes, one can identify colours, one does better object descriptions, etc. I'm still anxiously waiting for the "one app to rule them all" as it were. If you want to turn this project into a business, I would happily pay extra for features like OCR, bar code scanning, colour identification, etc, just to have everything in one app, and be able to use all the functions while only taking one photo. This is still an extremely knew space, so I think that every app still has a chance to become the overall winner.
A few questions:
1. Have you heard of, or tried, $1? I assume you're using a similar approach to them. How do you feel your app compares? I can (and will at some point) do a real-world test comparing them, but it's always nice to here the developer's thoughts.
2. Can your app do OCR? I didn't see that mentioned anywhere. But it's really nice to have both functions in a single app. Especially if I'm using it on photos; most photos today have captions, most products have writing on them, etc. So it's a big win to be able to get both functions in a single app, rather than having to take the same picture with two different apps. Right now, the only apps that offer both functions in one app are $1 (not free), and Microsoft's seeing AI app (not private).
I'm getting to a place where I feel like I need to have four different image identification apps installed, and it's kind of frustrating. One does better OCR, one can scan bar codes, one can identify colours, one does better object descriptions, etc. I'm still anxiously waiting for the "one app to rule them all" as it were. If you want to turn this project into a business, I would happily pay extra for features like OCR, bar code scanning, colour identification, etc, just to have everything in one app, and be able to use all the functions while only taking one photo. This is still an extremely knew space, so I think that every app still has a chance to become the overall winner.
taleser [OP] 2 points 6y ago
Thank you so much for the reply!
The timing is definitely a bummer, but it's exciting to see big companies invest a lot in accessibility technology.
At this point I've found that by overriding traditional VoiceOver gestures with simpler ones, I may have made Perigo less accessible than it should be. I sincerely apologize for any confusion that may cause if you do try it out.
Aipoly is a wonderful tool, and you are correct, our apps do work in a similar way by running neural networks right on your phone. Aipoly is great at object detection and I hope Perigo can provide a more general description of your photos and the things around you.
Those are really great suggestions, and I hope some day Perigo can be your all in one camera with OCR, barcodes, and more!
As of now, Seeing AI already has a tremendous amount of functionality, and is with no doubt the best solution available right now. Instead of competing against a tech behemoth, I think it would be really cool if Perigo Sight becomes an open source initiative, so that anyone can add new features and the community can help design the best app possible.
Thanks again for the advice!
The timing is definitely a bummer, but it's exciting to see big companies invest a lot in accessibility technology.
At this point I've found that by overriding traditional VoiceOver gestures with simpler ones, I may have made Perigo less accessible than it should be. I sincerely apologize for any confusion that may cause if you do try it out.
Aipoly is a wonderful tool, and you are correct, our apps do work in a similar way by running neural networks right on your phone. Aipoly is great at object detection and I hope Perigo can provide a more general description of your photos and the things around you.
Those are really great suggestions, and I hope some day Perigo can be your all in one camera with OCR, barcodes, and more!
As of now, Seeing AI already has a tremendous amount of functionality, and is with no doubt the best solution available right now. Instead of competing against a tech behemoth, I think it would be really cool if Perigo Sight becomes an open source initiative, so that anyone can add new features and the community can help design the best app possible.
Thanks again for the advice!
fastfinge 2 points 6y ago
> the best solution available
It isn't, unfortunately. If you read that privacy policy...if you wouldn't feel comfortable livestreaming it on youtube, don't put it in front of Seeing AI. Because you get pretty much no data protection at all. So I can't use it to OCR my mail (even though it can read documents) because I wouldn't know if it's my bank statement, junk mail, or a credit card bill, until it's too late! Instead, I need to use KNFB Reader for that. Plus, Seeing AI can't identify colours. I have to use AIPoly instead. So I have to flip regularly between three different apps. First world problems, I know! :-) At least all three apps exist and are all being improved. It's an exciting time to be blind.
> be really cool if Perigo Sight becomes an open source initiative,
That would be really cool! It seems to work for $1, and has allowed NVDA to become the best screen reader anywhere, on any platform. Though I wonder how many people can really contribute to code involving cutting edge AI like this? I'd be surprised if there were more than a hundred programmers currently living who can do this kind of work.
> if you do try it out.
I will! I did a post a few months ago giving a comparison between the OCR quality of Eye of Providence and KNFB Reader. It's time to do another one, comparing the new options, now that we have multiple object identification apps.
It isn't, unfortunately. If you read that privacy policy...if you wouldn't feel comfortable livestreaming it on youtube, don't put it in front of Seeing AI. Because you get pretty much no data protection at all. So I can't use it to OCR my mail (even though it can read documents) because I wouldn't know if it's my bank statement, junk mail, or a credit card bill, until it's too late! Instead, I need to use KNFB Reader for that. Plus, Seeing AI can't identify colours. I have to use AIPoly instead. So I have to flip regularly between three different apps. First world problems, I know! :-) At least all three apps exist and are all being improved. It's an exciting time to be blind.
> be really cool if Perigo Sight becomes an open source initiative,
That would be really cool! It seems to work for $1, and has allowed NVDA to become the best screen reader anywhere, on any platform. Though I wonder how many people can really contribute to code involving cutting edge AI like this? I'd be surprised if there were more than a hundred programmers currently living who can do this kind of work.
> if you do try it out.
I will! I did a post a few months ago giving a comparison between the OCR quality of Eye of Providence and KNFB Reader. It's time to do another one, comparing the new options, now that we have multiple object identification apps.
taleser [OP] 2 points 6y ago
It seems like the ultimate sight app has yet to be created! But over the next few years, apps with both versatile functionality and privacy will definitely become available. Hopefully, Perigo can be one of those apps.
At the moment, Perigo has a couple of limitations. The VoiceOver gestures on the current version are wonky, and will be fixed in an update tomorrow or the day after. Some good news is that there's a roadmap for offline OCR and facial recognition in the coming weeks!
I was wondering if you found the mode switching gestures on Seeing AI easy to use as compared to Aipoly's and if you have any suggestions for other high priority features to include in Perigo. I'll let you know when Perigo goes open source!
At the moment, Perigo has a couple of limitations. The VoiceOver gestures on the current version are wonky, and will be fixed in an update tomorrow or the day after. Some good news is that there's a roadmap for offline OCR and facial recognition in the coming weeks!
I was wondering if you found the mode switching gestures on Seeing AI easy to use as compared to Aipoly's and if you have any suggestions for other high priority features to include in Perigo. I'll let you know when Perigo goes open source!
fastfinge 1 points 6y ago
> I was wondering if you found the mode switching gestures on Seeing AI easy to use as compared to Aipoly's
They both work fine. I think that Seeing AI would be easier for low vision people, though, as it's just one list to flick through, rather than several controls.
> a roadmap for offline OCR
YAY!
> high priority features
I'll give you more of an idea tomorrow or the next day, once I've spent an hour or two with the app.
They both work fine. I think that Seeing AI would be easier for low vision people, though, as it's just one list to flick through, rather than several controls.
> a roadmap for offline OCR
YAY!
> high priority features
I'll give you more of an idea tomorrow or the next day, once I've spent an hour or two with the app.
1ird 1 points 6y ago
I don't have an iPhone but I'll give it a whirl on my girlfriend's when I get a chance. Any chance of an android release?
taleser [OP] 1 points 6y ago
Definitely possible, especially if there's enough interest! Hope you find it useful.
1ird 2 points 6y ago
I'll be back with my thoughts on it.
taleser [OP] 1 points 6y ago
That would be great. All feedback will help shape and direct future updates. Version 1.1 releasing tomorrow to fix Voice Over issue.
This nonprofit website is run by volunteers.
Please contribute if you can. Thank you!
Our mission is to provide everyone with access to large-
scale community websites for the good of humanity.
Without ads, without tracking, without greed.
scale community websites for the good of humanity.
Without ads, without tracking, without greed.
©2023 HumbleCat Inc • HumbleCat is a 501(c)3 nonprofit based in Michigan, USA.