Lynda Tang Direct Supervisor: Paul Medlock-Walton Faculty Supervisor: Hal Abelson UAP Project Writeup

Creating Customized User Interfaces for App Inventor Introduction: App Inventor is an open source Android application development environment that uses coding blocks instead of lines of script. By using colorful drag-and-drop blocks to represent programming logic, App Inventor abstracts away the complexities and specificities of programming languages and their respective syntaxes, and focuses on teaching users logical computational thinking strategies. Block-based programming is a good way to onboard and teach beginner programmers, as many find that the graphical representation of the blocks make it easier to use and understand than conventional scripting languages [2]. As of 2015, over 3 million people around the world currently use App Inventor, and the site has over 100,000 active weekly users. App Inventor can be used in a myriad of ways to create a variety of different applications. In addition to being a beginning programming language, App Inventor’s extensions and functions also support the construction of more complex applications that

can be paired with LEGO Mindstorms and Arduinos through the use of Internet and Bluetooth technology. App Inventor also supports other features such as: location based features, camera, touch interfaces, and various sensors. Many of App Inventor’s users pick up App Inventor outside of a formal learning environment by going through online walkthroughs and tutorials [4]. By using a blockbased programming language, App Inventor democratizes technology by lowering the bar of entry for people around the world. For my thesis, I plan to create and test a feature that allows for the formation of customized component and block sets in App Inventor.

Problem: In user experience and interface fields, there is an idea of “cognitive load” -- the amount of mental effort that is required by the user to operate the system [3]. In order to maximize the efficiency of the user, designers should seek to minimize the cognitive load users experience when using the interface. App Inventor’s current layout gives the user visual access to all the different functionalities that App Inventor provides. Although this type of display is suitable for experienced users, to beginner users and users who seek to focus on a specific functionality of App Inventor, it may be overwhelming and unintentionally increase user cognitive load.

Figure 1: The current visible components for all users of App Inventor

As demonstrated in Xie’s research on the skill progression of App Inventor users, after a certain point, user acquisition of App Inventor blocks and components begins to plateau, and we find that many users end up only using limited set of the App Inventor

features in their applications [4]. He hypothesizes that this could be due to the fact that only a small subset of the blocks is enough to create most of user applications, and I agree [4]. What this means is that the current App Inventor contains a lot of visual clutter in the forms of blocks and component types that the users seldom use. My thesis project will be about eliminating this extraneous cognitive load through the creation of customized sets for components and blocks. By limiting the amount of viewable elements, users can focus on the things that matter the most and can more efficiently learn and program with App Inventor.

Proposed Solution: The proposed solution I have is to create customized user interfaces with different palette and block sets that would suit different use purposes. Let’s take a look at some of the ways I hypothesize this feature would benefit current App Inventor user types: Beginner users: For beginner users, having a customized user interface allows users to only be exposed to components and blocks suitable for their level. By creating different sets that cater to user levels, we can prevent starting users from feeling overwhelmed by App Inventor’s options, and then gradually expose them to more and more features. In this scenario, customized user sets allows us to utilize “progressive disclosure”, a technique in which advanced features are put on a secondary screen so that users can focus on the basics. As pointed out by Nielsen, progressive disclosure “helps prioritize their (novice user’s) attention so that they only spend time on features that are most likely to be useful to them,” which in the long term can reduce mistakes and increase learning efficacy [1].

Specialized users: A specialized user is a user that uses App Inventor to develop for specific technologies, such as using Arduinos and Sensors for IOT applications. For these users, it would be also beneficial to apply the idea of “progressive disclosure”, only instead of catering blocks to skill level, we put the blocks that are relevant to their development purposes at the forefront. This way, they can easily navigate to the components and blocks they need. Tutorial Creators: By having a customized set function, we give tutorial creators the ability to create component and block sets that only feature the ones used in their tutorials. By doing so, learners can more easily find the components the tutorial uses and also remain focused on the task instead of getting distracted by other features.

Implementation: This project will be split into two parts: implementation and user testing: I plan to finish the implementation during the Fall semester and conduct user tests during the Spring semester. The implementation of this will mainly be done in React/Redux. I have picked up React/Redux in my previous work on creating a replica of the current App Inventor user interface in the new system, and will continue working on and modifying this replica throughout the first half of my thesis. The reason for refactoring App Inventor in React/Redux is that it allows us to more easily change and iterate on the system. At present, the Google Web Toolkit version of App Inventor is difficult to quickly modify, which limits the amount of experimentation we can do on the system’s user interface;

React/Redux offers us the flexibility required to create the customized sets needed for this project. The current work I have done so far on the project added various user interface features onto the React/Redux prototype:

Figure 2: React/Redux App Inventor Before/After

As shown in the images, my UAP work has allowed React/Redux App Inventor to more closely resemble the original App Inventor. Over the semester I mainly worked on the following parts:

1. Viewer Components: User interface components in the Viewer (the panel in between the Palette and the Components) are now displayed as they should be instead of placeholder blocks, but arrangement components were left as blocks on purpose for future debugging. As we can see, the Button and the PasswordTextbox are shown as buttons and password boxes respectively. The viewer also accurate reflects the properties of each component. Additionally, viewer components highlight when the user selects them in the Components panel. 2. Color/Image Properties Selector: Users can now pick from a set of predetermined colors and image assets when they want to change the visual appearance of components. When users click on ‘Choose Color’ or ‘Choose Image’ in the Properties panel, a modal pops up prompting them to pick their desired color/image. For example, the cat picture we see in the ‘After’ React/Redux App Inventor is the result of choosing an image asset from our selection. This feature will eventually be fleshed out so that users can upload images and use those as well, but this would require the use of a server. 3. Overall UI changes: I also made some minute UI changes overall that allow the React/Redux App Inventor to feel more like the original. This includes not only adjusting some of the layout and color scheme, but also creating a drop

down menu for screens and an accordion toggle in the Palette panel for the various component types. I will be building on top of this work for my MENG project, which also includes further updating the visual aesthetics and adding the customized sets feature. As App Inventor currently does not have the ability to display a subset of features, I plan to create a system that will allow customized palettes and their relevant blocks to show while placing the rest on a separate screen or in a drop down menu. There are plenty of software that utilize this model: for example, Photoshop lets users create their own workspace by selecting which windows to view under the ‘Window’ dropdown, and Microsoft Word places infrequently used toolbars under the ‘View’ dropdown header. The system should be able to handle the creation of a customized set from any subset of designated components/blocks, and should be able to allow for the editing and testing of these sets. Set creators can compile their custom sets by inputting a list of strings or checking off a list that represent the different component and block types. The system should then only display those component/blocks that the set creator has chosen and place the rest out of the user’s main view. When users view the limited components, they will only see the drop down labels that contain components that are in the limited set. For example, if none of the sensor components are included in the limited set, the user will not be able to see the ‘Sensor’ tab in Palette panel. An alternative to this form of display is to show all the limited components under one tab in the Palette panel; I believe, however, that this is a large enough deviation from the original App Inventor layout that it may confuse users who start off with the limited set who transition to using the full version of App Inventor. A

further consideration is to allow more than one accordion tab to be open at a time. This will increase the efficiency of users since they do not need to click open the tabs for access to various components, and since there is a limited set of components we do not need to worry about over-cluttering the Palette panel. The Block viewer will function similarly. When users open the block drawers, only the blocks that are in the block set are displayed, and drawers with no blocks in the block set in them will not be shown.

User Testing: In order to test the system, I plan to conduct two types of user tests: on novice users and on experienced users. The novice user test will be conducted with tutorials built for Youth Mobile Power in collaboration with Youth Radio. Youth Mobile Power is a project by Youth Radio that highlights young people who use mobile technology to aid their neighborhoods and communities. By lowering the barrier of entry and making it easier to create mobile applications, App Inventor is a suitable partner to Youth Mobile Power, and can support Youth Mobile Power’s goal in encouraging youths to create societychanging software by providing tutorials and features that make it easier for novices to pick up and learn App Inventor. The customized sets feature will be tested with the instructional material created in collaboration with this project, and this test will be conducted on users who have no prior exposure to App Inventor. I plan to separate the users into two groups: one which will use the full palette of App Inventor components and blocks and the other which will use a customized set catered to the tutorial they follow. In order to imitate the environment of most App Inventor learners, the users will

follow one of the Youth Mobile Power tutorials that will guide them through building a beginner application. Users will be encouraged to talk aloud during the testing and timed to see which process is more efficient. This will provide us with concrete data on whether or not having a limited set of blocks actually enables novice users to pick up and use App Inventor more easily. The experienced user test will be conducted on advanced App Inventor users, both in lab and outside. I hope that I will be able to find specialized users through contacting a few of the lab members who are developing with Internet of Things on the team, and be able to understand their opinions on having a more targeted App Inventor interface catered to their needs. Additionally, I also hope to find more experienced App Inventor users and have them try to use App Inventor with the customized sets. This test will allow us to see whether or not the sets in question are enough to fulfill the advanced user’s needs, and also whether or not users will be able to find elements that may not be in the component/block set they are originally given.

Projected Results: I hope this research will allow us to see the results of applying “progressive disclosure” on App Inventor’s user experience and improve the experience for current and new users of App Inventor. I hypothesize that having customized sets will increase the efficiency of users going through tutorials as they spend less time searching for the components that the tutorial calls for and can more easily grasp ideas and construct applications.

Previous Work/Research: Benjamin Xie has done prior research on App Inventor on the usability rate of App Inventor blocks and depth of capability of App Inventor users [4]. Xie’s research results provide several interesting insights that will aid in creating the component and block sets. One of the important parts of Xie’s results relevant to this project is the graph of block frequency to project number:

Figure 3: Graph of average number of blocks vs project number [5]

This graph tells us that beginner users (users with under 5 projects) typically usually use less than 10 blocks for their project, and even advanced users (users with more than 10 projects) use less than 20 blocks for their project. Xie concludes that this plateauing

pattern seen in the graph above supports the idea that users demonstrate capability through more advanced uses and combinations of the same blocks, rather than using many different components and blocks [4]. Customized user component and block sets would support the development of skills as having a limited set encourages users to utilize what they already know to create more sophisticated combinations, which Xie asserts is essential for learning retention [4]. Xie, in his thesis, also provides a three-phase model of perceived complexity in App Inventor. This model provides a starting platform in choosing which components/blocks are included at what levels should we decide to create predetermined user level sets. Xie also writes that many users of App Inventor pick up the software to create applications with a specific goal in mind, which provides us with another way of arranging the component and block sets, by application type [4].

Future Work: An extension of the project that goes beyond the minimum viable product is to create a system that allows users to assemble and distribute their own customized sets. This will let advanced users create a workspace that is personalized and customized to their needs. Additionally, a stretch goal would be to allow users to share these custom sets, which means that the users who create tutorials and walkthroughs can also provide supplementary premade sets to their readers, allowing educators to create targeted projects for their students. With this in mind, this project’s infrastructure should be flexible enough to allow people to share sets in the future.

Bibliography: [1] Nielsen, Jakob. "Nielsen Norman Group." Progressive Disclosure. Accessed May 22, 2017. https://www.nngroup.com/articles/progressive-disclosure/.

[2] Weintrop, D. and Wilensky, U. 2015. To Block or Not to Block, That is the Question: Students’ Perceptions of Blocks-based Programming. Proceedings of the 14th International Conference on Interaction Design and Children (New York, NY, USA), 199–208.

[3] Whitenton, Katheryn. "Nielsen Norman Group." Minimize Cognitive Load to Maximize Usability. Accessed May 22, 2017. https://www.nngroup.com/articles/minimize-cognitive-load/.

[4] Xie, Benjamin Xiang-Yu. "Progression of Computational Thinking Skills Demonstrated by App Inventor Users." Master's thesis, Massachusetts Institute of Technology, 2016. 2016. https://drive.google.com/file/d/0B3jsksMcCW5bZGVuemF5VDQ0QWc/view.