{"id":1571,"date":"2015-10-11T20:53:33","date_gmt":"2015-10-11T19:53:33","guid":{"rendered":"http:\/\/quantum-bits.org\/?p=1571"},"modified":"2022-08-12T17:26:58","modified_gmt":"2022-08-12T16:26:58","slug":"1571","status":"publish","type":"post","link":"https:\/\/www.quantum-bits.org\/?p=1571","title":{"rendered":"My week-end with E.V.E"},"content":{"rendered":"<p>I spent part of my week-end finishing the voice recognition sub-system, tweaking E.V.E&#8217;s UX and cleaning up E.V.E \/ A.D.A.M code.<\/p>\n<p>Let&#8217;s see how it went !<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1297\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/09\/rasbperrypi-hacks.jpg\" alt=\"rasbperrypi-hacks\" width=\"985\" height=\"503\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/09\/rasbperrypi-hacks.jpg 985w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/09\/rasbperrypi-hacks-300x153.jpg 300w\" sizes=\"(max-width: 985px) 100vw, 985px\" \/><\/p>\n<p><strong>E.V.E loves snakes<\/strong><\/p>\n<p>E.V.E&#8217;s interface is based on HTML5\/Javascript and runs inside a browser. <a href=\"http:\/\/quantum-bits.org\/?p=1361\" target=\"_blank\" rel=\"noopener\">I wasn&#8217;t too happy<\/a> with my RPi&#8217;s browsers until I settled for <a href=\"https:\/\/wiki.gnome.org\/Apps\/Web\" target=\"_blank\" rel=\"noopener\">Epiphany<\/a>. Even then, I had to write a little script to simulate a <code>F11<\/code> key press to run the browser in fullscreen mode.<\/p>\n<p>And actually I didn&#8217;t really need a full browser. Since I was happy with Epiphany&#8217;s <a href=\"http:\/\/webkitgtk.org\/\" target=\"_blank\" rel=\"noopener\">webkit<\/a>, I decided to strip E.V.E to a bare minimum (so to speak &#8230;).<\/p>\n<p>I wrote down a few lines of python and used the WebKit python binding to create a simple web view, the size of my RPi touchscreen (800 x 480):<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1579\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-python.png\" alt=\"eve-python\" width=\"660\" height=\"566\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-python.png 660w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-python-300x257.png 300w\" sizes=\"(max-width: 660px) 100vw, 660px\" \/><\/p>\n<p>Simple and effective. It runs E.V.E perfectly well and is much faster to launch.<\/p>\n<p>The only drawback was the annoying presence of the Window Manager (<a href=\"http:\/\/openbox.org\/wiki\/Main_Page\" target=\"_blank\" rel=\"noopener\">OpenBox<\/a>) decorations around the window. So, I edited&nbsp;<code>~\/.config\/openbox\/lxde-pi-rc.xml<\/code> and added these lines to remove all window decorations for &#8220;eve.py&#8221;:<\/p>\n<p style=\"padding-left: 60px;\"><code>&lt;application name=\"eve.py\"&gt;<br \/>\n&nbsp;&nbsp;&lt;decor&gt;no&lt;\/decor&gt;<br \/>\n&lt;\/application&gt;<br \/>\n<\/code><\/p>\n<p>E.V.E now boots up much faster, and the unnecessary clutter of a full browser is&nbsp;now removed.<\/p>\n<p><b>Tweaking E.V.E&#8217;s interface<\/b><\/p>\n<p>After a couple of weeks playing with E.V.E&#8217;s touch interface, I felt I had to tweak its UX a bit:<\/p>\n<ul>\n<li>I removed the &#8220;home button&#8221; for E.V.E home screen (with was useless) and dimmed it&#8217;s color a bit on other apps (Orange TV and Internet Radio)<\/li>\n<li>I removed the &#8220;information button&#8221; with was too small and quite useless<\/li>\n<li>The &#8220;configuration button&#8221; is much bigger (and thus more accessible). It now occupies the bottom left corner<\/li>\n<li>I added a switch button &#8220;On \/ Off&#8221; on the top left corner. When touched, the screen turns black (and returns &#8220;On&#8221; when the screen is touched back)<\/li>\n<\/ul>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1580\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/home-screen-ux-0.0.5.png\" alt=\"home-screen-ux-0.0.5\" width=\"918\" height=\"511\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/home-screen-ux-0.0.5.png 918w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/home-screen-ux-0.0.5-300x167.png 300w\" sizes=\"(max-width: 918px) 100vw, 918px\" \/><\/p>\n<p><strong>Notification system and error controls<\/strong><\/p>\n<p>Now that I am in the process of integrating voice recognition, I need a notification system to display contextual feedback. I settled on a non intrusive &#8220;toast&#8221; <a href=\"https:\/\/en.wikipedia.org\/wiki\/Pop-up_notification\" target=\"_blank\" rel=\"noopener\">notifications<\/a>. I created 5 different kind of toasts:<\/p>\n<ul>\n<li>Success feedback and messages<\/li>\n<li>Warning messages<\/li>\n<li>Error messages<\/li>\n<li>Information messages<\/li>\n<li>Responses<\/li>\n<\/ul>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1582\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/notifications.png\" alt=\"notifications\" width=\"641\" height=\"295\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/notifications.png 641w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/notifications-300x138.png 300w\" sizes=\"(max-width: 641px) 100vw, 641px\" \/><\/p>\n<p>The first 4 ones are displayed on the top, while the last one is displayed on the bottom and occupies the full width of the display. It will be used for answers from the voice recognition system. The &#8220;success&#8221;, &#8220;warning&#8221;, &#8220;error&#8221; and &#8220;information&#8221; toasts disappear after a few seconds (or when touched). The &#8220;response&#8221; toast stays up&nbsp;until the user touches it.<\/p>\n<p>The &#8220;error message toasts&#8221; allowed me to implement a few error controls that were dramatically missing. Here&#8217;s what it looks like (when I kill the A.D.A.M process):<\/p>\n<p><center><div style=\"width: 800px;\" class=\"wp-video\"><!--[if lt IE 9]><script>document.createElement('video');<\/script><![endif]-->\n<video class=\"wp-video-shortcode\" id=\"video-1571-1\" width=\"800\" height=\"480\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-toaster.mp4?_=1\" \/><a href=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-toaster.mp4\">http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-toaster.mp4<\/a><\/video><\/div><\/center><\/p>\n<p><strong>Orange TV API<\/strong><\/p>\n<p>The Orange TV application has been split into two parts:<\/p>\n<ul>\n<li>The Orange TV application itself<\/li>\n<li>An API that can be used by the voice recognition system<\/li>\n<\/ul>\n<p><strong>A.D.A.M<\/strong><\/p>\n<p>The&nbsp;E.V.E \/ A.D.A.M \/ wit.ai workflow has been implemented :<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1540\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-adam-voice.png\" alt=\"eve-adam-voice\" width=\"657\" height=\"453\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-adam-voice.png 657w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-adam-voice-300x207.png 300w\" sizes=\"(max-width: 657px) 100vw, 657px\" \/><\/p>\n<p>Since I&#8217;m using the biggest of my two USB microphones, the trimming \/ filtering process it not necessary. Actually, to speed up the exchanges between A.D.A.M and wit.ai, I downgraded the record quality to 8 bits \/ 8000 Hz \/ mono (instead &nbsp;of 16 bits \/ 48000 Hz \/ stereo).<\/p>\n<p><strong>wit.ai configuration and training<\/strong><\/p>\n<p>I was impatient to test the full recognition process, so I spent&nbsp;only a few moments on wit.ai&#8217;s&nbsp;configuration and training. I created only two intents (and no entity yet):<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1585\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox_.png\" alt=\"wit.ai.inbox\" width=\"938\" height=\"462\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox_.png 938w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox_-300x148.png 300w\" sizes=\"(max-width: 938px) 100vw, 938px\" \/><\/p>\n<p>With so few intents, it means that, for now, the parsing process relies too much on E.V.E&#8217;s code. I&#8217;ll have to spend more time on wit.ai later on, and simplify E.V.E&#8217;s code afterward.<\/p>\n<p>I tried a few recognitions and trained wit.ai on these (mostly, droping false recognitions and connecting to the proper&nbsp;intents):<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1586\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox2_.png\" alt=\"wit.ai.inbox2\" width=\"967\" height=\"741\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox2_.png 967w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/wit.ai_.inbox2_-300x230.png 300w\" sizes=\"(max-width: 967px) 100vw, 967px\" \/><\/p>\n<p><strong>Wrapping things up<\/strong><\/p>\n<p>It&#8217;s now time to wrap things up and test E.V.E brand new capabilities:<\/p>\n<p><center><div style=\"width: 568px;\" class=\"wp-video\"><video class=\"wp-video-shortcode\" id=\"video-1571-2\" width=\"568\" height=\"320\" preload=\"metadata\" controls=\"controls\"><source type=\"video\/mp4\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-0.0.5.mp4?_=2\" \/><a href=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-0.0.5.mp4\">http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/eve-0.0.5.mp4<\/a><\/video><\/div><\/center><\/p>\n<p>I must say that I&#8217;m very happy with it. It&#8217;s a bit slow (at least, slower than Siri or Cortana, or whatever is Google&#8217;s name for its Android speech recognition system), but it is working nicely. And I can&#8217;t turn my TV on and off with Siri anyway !<\/p>\n<p>I created a Git repository, and I will push E.V.E \/ A.D.AM&#8217;s code soon (a bit of cleaning yet to do).<\/p>\n<p>I was testing E.V.E from my kitchen&#8217;s bar. It gave me ideas. Let&#8217;s enjoy the moment and have a treat with a nice home-made pizza !<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-full wp-image-1578\" src=\"http:\/\/quantum-bits.org\/wp-content\/uploads\/2015\/10\/pizza-time.jpg\" alt=\"pizza-time\" width=\"1280\" height=\"786\" srcset=\"https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/pizza-time.jpg 1280w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/pizza-time-300x184.jpg 300w, https:\/\/www.quantum-bits.org\/wp-content\/uploads\/2015\/10\/pizza-time-1024x629.jpg 1024w\" sizes=\"(max-width: 1280px) 100vw, 1280px\" \/><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I spent part of my week-end finishing the voice recognition sub-system, tweaking E.V.E&#8217;s UX and cleaning up E.V.E \/ A.D.A.M code. Let&#8217;s see how it went ! E.V.E loves snakes &#8230;<\/p>\n","protected":false},"author":1,"featured_media":3851,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0},"categories":[21],"tags":[],"_links":{"self":[{"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/posts\/1571"}],"collection":[{"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1571"}],"version-history":[{"count":0,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/posts\/1571\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=\/wp\/v2\/media\/3851"}],"wp:attachment":[{"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1571"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1571"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.quantum-bits.org\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1571"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}