Search

Search Results

• 1. [Article] How well do professional developers test with code coverage visualizations? An empirical study

  "Code coverage visualizations using block coverage neither guided developers toward productive testing strategies, nor did these visualizations motivate developers to write more tests or help them find ...

  Citation × Citation Citation Abstract Copy Title: How well do professional developers test with code coverage visualizations? An empirical study Author: Burnett, Margaret, 1949-, Lawrance, Joseph A., Clarke, Steven, Oregon State University. Dept. of Computer Science, Rothermel, Gregg "Code coverage visualizations using block coverage neither guided developers toward productive testing strategies, nor did these visualizations motivate developers to write more tests or help them find more faults than the control group. Nevertheless, code coverage visualizations did influence developers in a few important ways. Code coverage visualizations led developers to overestimate their test effectiveness more than the control group. Yet, these same visualizations reduced the variability in the number of test cases developers wrote by changing the standard developers used to evaluate their test effectiveness. Thus, the true power of testing visualizations lies not only with the faults that visualizations can highlight; it also lies in how visualizations can change how developers think about testing. Testing visualizations guide developers toward a particular standard of effectiveness, so if we want developers to test software adequately, we must ensure that the coverage criteria we choose to visualize leads developers toward a good standard of test effectiveness."--Conclusion Title: How well do professional developers test with code coverage visualizations? An empirical study Author: Burnett, Margaret, 1949-, Lawrance, Joseph A., Clarke, Steven, Oregon State University. Dept. of Computer Science, Rothermel, Gregg "Code coverage visualizations using block coverage neither guided developers toward productive testing strategies, nor did these visualizations motivate developers to write more tests or help them find more faults than the control group. Nevertheless, code coverage visualizations did influence developers in a few important ways. Code coverage visualizations led developers to overestimate their test effectiveness more than the control group. Yet, these same visualizations reduced the variability in the number of test cases developers wrote by changing the standard developers used to evaluate their test effectiveness. Thus, the true power of testing visualizations lies not only with the faults that visualizations can highlight; it also lies in how visualizations can change how developers think about testing. Testing visualizations guide developers toward a particular standard of effectiveness, so if we want developers to test software adequately, we must ensure that the coverage criteria we choose to visualize leads developers toward a good standard of test effectiveness."--Conclusion Close

• 2. [Article] On to the real world : gender and self-efficacy in Excel

  Although there have been a number of studies of end-user software development tasks, few of them have considered gender issues for real end-user developers in real-world environments for end-user programming. In ...

  Citation × Citation Citation Abstract Copy Title: On to the real world : gender and self-efficacy in Excel Author: Inman, Derek, Rector, Kyle, Beckwith, Laura A., Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science Although there have been a number of studies of end-user software development tasks, few of them have considered gender issues for real end-user developers in real-world environments for end-user programming. In order to be trusted, the results of such laboratory studies must always be re-evaluated with fewer controls, more closely reflecting real-world conditions. Therefore, the research question in this paper is whether the results of a Gender HCI controlled study generalize -- to real-world end-user developers, in a real-world spreadsheet environment, using a realworld spreadsheet. Our findings are that the concepts revealed by the original laboratory study appear to be quite robust, being demonstrated in multiple ways in this real-world environment. Title: On to the real world : gender and self-efficacy in Excel Author: Inman, Derek, Rector, Kyle, Beckwith, Laura A., Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science Although there have been a number of studies of end-user software development tasks, few of them have considered gender issues for real end-user developers in real-world environments for end-user programming. In order to be trusted, the results of such laboratory studies must always be re-evaluated with fewer controls, more closely reflecting real-world conditions. Therefore, the research question in this paper is whether the results of a Gender HCI controlled study generalize -- to real-world end-user developers, in a real-world spreadsheet environment, using a realworld spreadsheet. Our findings are that the concepts revealed by the original laboratory study appear to be quite robust, being demonstrated in multiple ways in this real-world environment. Close

• 3. [Article] Workshop report : from end-user programming to end-user software engineering (a CHI'06) Workshop)

  End users create software when they use spreadsheet systems, web authoring tools and graphical languages, and when they create educational simulations, macros-by-demonstration, and dynamic e-business web ...

  Citation × Citation Citation Abstract Copy Title: Workshop report : from end-user programming to end-user software engineering (a CHI'06) Workshop) Author: Myers, Brad A., Burnett, Margaret, 1949-, Rosson, Mary Beth, Oregon State University. Dept. of Computer Science, Wiedenbeck, Susan, Leibel, Adam End users create software when they use spreadsheet systems, web authoring tools and graphical languages, and when they create educational simulations, macros-by-demonstration, and dynamic e-business web applications and mash-ups. Some end-user developers, such as accountants or teachers, may have no formal training at all in programming. Others, such as scientists or sysadmins, may have significant background in programming, but they still do not see their goals as producing software: rather, they see software as a means to some other goal, such as visualizing a scientific phenomenon or getting a computer set up so that it can talk to the printers in the building. It has become well known that errors are pervasive in end-user software, and the resulting impact is sometimes enormous. A growing number of researchers and developers are working on ways to make the software created by end users more reliable, by looking beyond the "create" aspect of the software lifecycle. The WEUSE workshop (Workshop on End-User Software Engineering), organized by Margaret Burnett, Brad Myers, Mary Beth Rosson, and Susan Wiedenbeck for CHI'06, brought together academic and industrial researchers who are addressing this topic with industry representatives who are deploying end-user programming systems. The objective was to facilitate sharing of real-world problems and solutions. Four real-world end-user software engineering issues formed the focus of the CHI'06 workshop: extending an open source paradigm for end-user developers of MATLAB programs, helping disparate audiences learn to use the Flash end-user programming product, dealing with mission-critical aspects of end-user programming situations that occur in customization of healthcare systems, and supporting the high-pressure development done by sysadmins. Title: Workshop report : from end-user programming to end-user software engineering (a CHI'06) Workshop) Author: Myers, Brad A., Burnett, Margaret, 1949-, Rosson, Mary Beth, Oregon State University. Dept. of Computer Science, Wiedenbeck, Susan, Leibel, Adam End users create software when they use spreadsheet systems, web authoring tools and graphical languages, and when they create educational simulations, macros-by-demonstration, and dynamic e-business web applications and mash-ups. Some end-user developers, such as accountants or teachers, may have no formal training at all in programming. Others, such as scientists or sysadmins, may have significant background in programming, but they still do not see their goals as producing software: rather, they see software as a means to some other goal, such as visualizing a scientific phenomenon or getting a computer set up so that it can talk to the printers in the building. It has become well known that errors are pervasive in end-user software, and the resulting impact is sometimes enormous. A growing number of researchers and developers are working on ways to make the software created by end users more reliable, by looking beyond the "create" aspect of the software lifecycle. The WEUSE workshop (Workshop on End-User Software Engineering), organized by Margaret Burnett, Brad Myers, Mary Beth Rosson, and Susan Wiedenbeck for CHI'06, brought together academic and industrial researchers who are addressing this topic with industry representatives who are deploying end-user programming systems. The objective was to facilitate sharing of real-world problems and solutions. Four real-world end-user software engineering issues formed the focus of the CHI'06 workshop: extending an open source paradigm for end-user developers of MATLAB programs, helping disparate audiences learn to use the Flash end-user programming product, dealing with mission-critical aspects of end-user programming situations that occur in customization of healthcare systems, and supporting the high-pressure development done by sysadmins. Close

• 4. [Article] An integrated testing and fault localization methodology for spreadsheet languages

  Spreadsheet languages, which include commercial spreadsheets and various research systems, have proven to be flexible tools in many settings. Research shows, however, that spreadsheets often contain faults. ...

  Citation × Citation Citation Abstract Copy Title: An integrated testing and fault localization methodology for spreadsheet languages Author: Burnett, Margaret, 1949-, Reichwein, James, Oregon State University. Dept. of Computer Science Spreadsheet languages, which include commercial spreadsheets and various research systems, have proven to be flexible tools in many settings. Research shows, however, that spreadsheets often contain faults. This thesis presents an integrated testing and fault localization methodology for spreadsheets. This methodology allows spreadsheet developers to engage in modeless development, testing and debugging activities. Furthermore, we provide an interface to our methodology that does not require an understanding of testing and debugging theory. To accomplish this, we introduce the notion of fault likelihood: the likelihood that a given cell contains a fault that contributes to an known failure in the spreadsheet. To estimate fault likelihood we present five properties that we feel should govern its behavior. We then discuss our implementation of this methodology and illustrate its use. Title: An integrated testing and fault localization methodology for spreadsheet languages Author: Burnett, Margaret, 1949-, Reichwein, James, Oregon State University. Dept. of Computer Science Spreadsheet languages, which include commercial spreadsheets and various research systems, have proven to be flexible tools in many settings. Research shows, however, that spreadsheets often contain faults. This thesis presents an integrated testing and fault localization methodology for spreadsheets. This methodology allows spreadsheet developers to engage in modeless development, testing and debugging activities. Furthermore, we provide an interface to our methodology that does not require an understanding of testing and debugging theory. To accomplish this, we introduce the notion of fault likelihood: the likelihood that a given cell contains a fault that contributes to an known failure in the spreadsheet. To estimate fault likelihood we present five properties that we feel should govern its behavior. We then discuss our implementation of this methodology and illustrate its use. Close

• 5. [Article] Interactive, visual fault localization support for end-user programmers

  End-user programmers are writing an unprecedented number of programs, primarily using languages and environments that incorporate a number of interactive and visual programming techniques. To help these users ...

  Citation × Citation Citation Abstract Copy Title: Interactive, visual fault localization support for end-user programmers Author: Cook, Curtis R., Burnett, Margaret, 1949-, Reichwein, James, Prabhakararao, Shrinu, Oregon State University. Dept. of Computer Science, Ruthruff, Joseph Ronald, Creswick, Eugene Rogan End-user programmers are writing an unprecedented number of programs, primarily using languages and environments that incorporate a number of interactive and visual programming techniques. To help these users debug these programs, we have developed an entirely visual, interactive approach to fault localization. This paper presents the approach. We also present the results of a think-aloud study that examined the interactive, human-centric issues that arise in end-user debugging using a fault localization strategy. Our results provide insights into the contributions such strategies can make to the end-user debugging process. Title: Interactive, visual fault localization support for end-user programmers Author: Cook, Curtis R., Burnett, Margaret, 1949-, Reichwein, James, Prabhakararao, Shrinu, Oregon State University. Dept. of Computer Science, Ruthruff, Joseph Ronald, Creswick, Eugene Rogan End-user programmers are writing an unprecedented number of programs, primarily using languages and environments that incorporate a number of interactive and visual programming techniques. To help these users debug these programs, we have developed an entirely visual, interactive approach to fault localization. This paper presents the approach. We also present the results of a think-aloud study that examined the interactive, human-centric issues that arise in end-user debugging using a fault localization strategy. Our results provide insights into the contributions such strategies can make to the end-user debugging process. Close

• 6. [Article] End-user testing for the Lyee methodology using the screen transition paradigm and WYSIWYT

  End-user specification of Lyee programs is one goal envisioned by the Lyee methodology. But with any software development effort comes the possibility of faults. Thus, providing end users a means to enter ...

  Citation × Citation Citation Abstract Copy Title: End-user testing for the Lyee methodology using the screen transition paradigm and WYSIWYT Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Brown, Darren, Rothermel, Greg End-user specification of Lyee programs is one goal envisioned by the Lyee methodology. But with any software development effort comes the possibility of faults. Thus, providing end users a means to enter their own specifications is not enough; they must also be provided with the means to find faults in their specifications, in a manner that is appropriate not only for the end user's programming environment but also for his or her background. In this paper, we present an approach to solving this problem that marries two proven technologies for end users. One methodology for enabling end users to program is the screen transition paradigm. One useful visual testing methodology is "What you see is what you test (WYSIWYT)". In this paper, we show that WYSIWYT test adequacy criteria can be used with the screen transition paradigm, and present a systematic translation from this paradigm to the formal model underlying WYSIWYT. Title: End-user testing for the Lyee methodology using the screen transition paradigm and WYSIWYT Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Brown, Darren, Rothermel, Greg End-user specification of Lyee programs is one goal envisioned by the Lyee methodology. But with any software development effort comes the possibility of faults. Thus, providing end users a means to enter their own specifications is not enough; they must also be provided with the means to find faults in their specifications, in a manner that is appropriate not only for the end user's programming environment but also for his or her background. In this paper, we present an approach to solving this problem that marries two proven technologies for end users. One methodology for enabling end users to program is the screen transition paradigm. One useful visual testing methodology is "What you see is what you test (WYSIWYT)". In this paper, we show that WYSIWYT test adequacy criteria can be used with the screen transition paradigm, and present a systematic translation from this paradigm to the formal model underlying WYSIWYT. Close

• 7. [Article] The impact of two orthogonal factors in interactive fault localization

  End users develop more software than any other group of programmers, using software authoring devices such as email filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite ...

  Citation × Citation Citation Abstract Copy Title: The impact of two orthogonal factors in interactive fault localization Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Ruthruff, Joseph Ronald, Rothermel, Gregg End users develop more software than any other group of programmers, using software authoring devices such as email filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been only a little research on finding ways to help these programmers with the dependability of the software they create. We have been working to address this problem in several ways, one of which includes supporting end-user debugging activities through fault localization techniques. Our preliminary studies have shown that some fault localization techniques can provide end users with effective feedback; however, these studies provide no guidance into the factors that impact technique effectiveness. This paper presents the results of an empirical study that examines the impact of two orthogonal factors on the effectiveness of fault localization techniques. Our results have implications for fault localization for end users, and highlight the importance of separately evaluating each factor in a fault localization technique. Title: The impact of two orthogonal factors in interactive fault localization Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Ruthruff, Joseph Ronald, Rothermel, Gregg End users develop more software than any other group of programmers, using software authoring devices such as email filtering editors, by-demonstration macro builders, and spreadsheet environments. Despite this, there has been only a little research on finding ways to help these programmers with the dependability of the software they create. We have been working to address this problem in several ways, one of which includes supporting end-user debugging activities through fault localization techniques. Our preliminary studies have shown that some fault localization techniques can provide end users with effective feedback; however, these studies provide no guidance into the factors that impact technique effectiveness. This paper presents the results of an empirical study that examines the impact of two orthogonal factors on the effectiveness of fault localization techniques. Our results have implications for fault localization for end users, and highlight the importance of separately evaluating each factor in a fault localization technique. Close

• 8. [Article] Steering programs via time travel

  Despite years of research into human computer interaction (HCI), the environments programmers must use for problem-solving today—with separate modes and tools for writing, compiling, testing, visualizing, ...

  Citation × Citation Citation Abstract Copy Title: Steering programs via time travel Author: Atwood, John Wesley, Jr., Yang, Sherry, Burnett, Margaret, 1949-, Walpole, Rebecca A., Oregon State University. Dept. of Computer Science, Wilcox, E. M. Despite years of research into human computer interaction (HCI), the environments programmers must use for problem-solving today—with separate modes and tools for writing, compiling, testing, visualizing, and debugging— derive their basic structure from historical accident, and take little advantage of HCI research into the cognitive issues of programming. Neglecting these issues is an impediment to the programmers' ability to produce reliable, maintainable software. In this paper, we describe a system in which programmers can modelessly steer as they specify, visualize, explore, and alter the behavior of a program while traveling through the program's logical time. This approach supports two often-neglected cognitive principles that programmers need for problemsolving. Title: Steering programs via time travel Author: Atwood, John Wesley, Jr., Yang, Sherry, Burnett, Margaret, 1949-, Walpole, Rebecca A., Oregon State University. Dept. of Computer Science, Wilcox, E. M. Despite years of research into human computer interaction (HCI), the environments programmers must use for problem-solving today—with separate modes and tools for writing, compiling, testing, visualizing, and debugging— derive their basic structure from historical accident, and take little advantage of HCI research into the cognitive issues of programming. Neglecting these issues is an impediment to the programmers' ability to produce reliable, maintainable software. In this paper, we describe a system in which programmers can modelessly steer as they specify, visualize, explore, and alter the behavior of a program while traveling through the program's logical time. This approach supports two often-neglected cognitive principles that programmers need for problemsolving. Close

• 9. [Article] Generalizing WYSIWYT visual testing to screen transition languages

  How can rigorous forms of testing be supported in a way that is both compatible with the visual aspect of visual programming languages, and usable by the audiences using those languages -- even when the ...

  Citation × Citation Citation Abstract Copy Title: Generalizing WYSIWYT visual testing to screen transition languages Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Brown, Darren, Rothermel, Greg How can rigorous forms of testing be supported in a way that is both compatible with the visual aspect of visual programming languages, and usable by the audiences using those languages -- even when the audience has no background in software engineering? Visual programs are likely to contain at least some errors, and supporting a visual form of testing would give users a way to spot those errors early in the program’s life. In previous work, we have developed a visual testing methodology known as WYSIWYT, for use in visual spreadsheet languages, and in this work, we show formally that this methodology can be generalized to screen transition diagrams. The algorithms and accompanying proof of the coverage equivalence that they ensure provide the mechanisms needed for the screen transition paradigm to incorporate WYSIWYT testing for both professional and end-user programming audiences. Title: Generalizing WYSIWYT visual testing to screen transition languages Author: Burnett, Margaret, 1949-, Oregon State University. Dept. of Computer Science, Brown, Darren, Rothermel, Greg How can rigorous forms of testing be supported in a way that is both compatible with the visual aspect of visual programming languages, and usable by the audiences using those languages -- even when the audience has no background in software engineering? Visual programs are likely to contain at least some errors, and supporting a visual form of testing would give users a way to spot those errors early in the program’s life. In previous work, we have developed a visual testing methodology known as WYSIWYT, for use in visual spreadsheet languages, and in this work, we show formally that this methodology can be generalized to screen transition diagrams. The algorithms and accompanying proof of the coverage equivalence that they ensure provide the mechanisms needed for the screen transition paradigm to incorporate WYSIWYT testing for both professional and end-user programming audiences. Close

• 10. [Article] Algorithm animation in a declarative visual programming language

  How might capabilities for algorithm animation be seamlessly integrated into a programming language that is both visual and declarative? Until now, visual programming language researchers have not attempted ...

  Citation × Citation Citation Abstract Copy Title: Algorithm animation in a declarative visual programming language Author: Burnett, Margaret, 1949-, Carlson, Paul M., Oregon State University. Dept. of Computer Science How might capabilities for algorithm animation be seamlessly integrated into a programming language that is both visual and declarative? Until now, visual programming language researchers have not attempted to answer that question, making the fruits of algorithm animation available only to users of textual progranming languages. Users of visual programming languages (VPLs) have been deprived of the unique semantic insights algorithm animation offers, insights that would foster the understanding and debugging of visual programs. We have answered the question by seamlessly integrating algorithm animation capabilities into Forms/3, a general-purpose, declarative VPL. Our results show that such a VPL can support algorithm animation without leaving the declarative, visual model, without adding new concepts to the language or how to program in it, and without deviating from the uniform representation established for the language. In addition, our research shows that the characteristics of declarative VPLs result in some interesting algorithm animation features not found in other systems. Title: Algorithm animation in a declarative visual programming language Author: Burnett, Margaret, 1949-, Carlson, Paul M., Oregon State University. Dept. of Computer Science How might capabilities for algorithm animation be seamlessly integrated into a programming language that is both visual and declarative? Until now, visual programming language researchers have not attempted to answer that question, making the fruits of algorithm animation available only to users of textual progranming languages. Users of visual programming languages (VPLs) have been deprived of the unique semantic insights algorithm animation offers, insights that would foster the understanding and debugging of visual programs. We have answered the question by seamlessly integrating algorithm animation capabilities into Forms/3, a general-purpose, declarative VPL. Our results show that such a VPL can support algorithm animation without leaving the declarative, visual model, without adding new concepts to the language or how to program in it, and without deviating from the uniform representation established for the language. In addition, our research shows that the characteristics of declarative VPLs result in some interesting algorithm animation features not found in other systems. Close