Showing posts with label UI. Show all posts
Showing posts with label UI. Show all posts
2008-11-21
Oblong Industries
About once a year I see something like this: http://oblong.com/ and it renews the passion I have for technology changing the way we interact with our machine creations. I only dream that someday I can create a work that is as visceral in its impact. I hope systems like this will one day be everywhere and also not require any UI gadgetry such as VR gloves like the ones shown in the video.
2008-11-03
It is all UI even API
It is all UI and UI leads us down certain paths.
This is an idea I've had for a long time but it hasn't been posted here before so let me lay down for you the why behind the tag line of this blog: "The power of software is the power of thoughts and ideas."
These are important concepts: The purpose of the UI is to aid the user in thinking about what they are doing inside their machine. The UI should match what they are thinking about as closely as possible.
This is all well and good for users. It gives them pretty looking sites that are easy to use and understand. It leads to ideas about making user customization of content a certain flavor of easy that involves allowing for graphical interaction with the system.
But, the programming language is UI too.
I have recently graduated into a tier of programmers that write programs for other programmers to use. But, I have always known that unless you are building the system from nothing then you are the user of another person's UI. It may be called an API but it is still UI.
Programming languages are UI too, they are UI with a limited audience, developers. Just as EE folks design CPU with a "programmer's model" in mind so too should library developers develop software with application developers in mind.
The best way to do this is through "dog-fooding" which is to say that a developer should be forced to eat their own dog-food. When you develop something you should have to use it. If you can't use it you can't really make well informed decisions about how that product should work.
As a developer I try to get inside the minds of the people who use my products. I try to anticipate what they will do, how they will do it, and what ways they may want to do things differently. I can operate this way because after nearly two decades of working with computers in one fashion or another I am convinced there is no right or wrong intrinsic in the machines themselves.
Any meaning, purpose, or sense of working or brokenness is determined by the user. To make a system that is bug free you have to know the user. The technology therefore must be designed from the user back to the machine.
The problem is compounded by the fact that there are multiple types of users. You can build classifications for each of them profiling what their expectations are and building their view of the system around that. Working for one user is broken for another. To design a truly beautiful system you need to account for as many of your user classes as possible.
Often what system designers and developers leave out of their design consideration is themselves. Developers are users too. The must live with their own design choices over time and are prone to the same flaws that other user classifications are prone to as well.
The reasons technologies succeed and fail are tied up in the intersections of actual functionality and perceived function and value. These are emotional decisions as much as logical ones. Very technically speaking any Turing complete language is capable of any task that any other Turing complete language can do.
So the power we find in expressive software is actually the ability to cleanly express some thought or idea. We are attempting to communicate both to the machine and the user some expression of action, computation, art, or logic that will achieve the goal of the user. The software is most effective when the developer and the user understand each other through the lens of the software.
This is true for your web application, mobile application, programming language, or your developer API.
The web framework developer is working for their user which is the web application developer. The persistence layer developer is working for their user which is both the framework developer and the application developer. The application developer is working for the end users of their system.
The most effective developers will be the ones that realize this and deliberately cross as many disciplines as appropriate for their application space. This kind of renaissance developer will be most effective at knowing how what they will write will be received. This kind of developer will know implicitly that it is all about user interface and developers are users too.
Just as architects have known for generations that a building must be beautiful both on the outside and the inside yet still respect the laws of nature so too must computing grow up. So in my phrase "the power of software is the power of thoughts and ideas" is nestled the concept that not only power but beauty and value of software is the beauty and value of thoughts and ideas.
The metrics of our beauty in software is the cognitive dissonance we create in attempting to enable our users to create their art with our tools. When you create software you are implicitly directing the thoughts of another person or machine. Just as an Orwellian language could forbade certain thoughts your API could forbid a design or direct users down a path. The challenge is to match your design to the users' thoughts and direct them toward good solutions without imposing a mental fascism.
The converse is side of this is that you want to direct your users toward maintainable and workable designs. The goal should be to guide not to force. So working out an good design is a balancing act.
Just some thoughts that I've had while looking at possible improvements to an existing development framework.
Edit: Oh yes, I saw this shiny "follow" gadget and added it to the upper right hand corner. Won't you do me a favor and click it a few dozen times so I don't feel too bad about having "0" followers. Thanks.
This is an idea I've had for a long time but it hasn't been posted here before so let me lay down for you the why behind the tag line of this blog: "The power of software is the power of thoughts and ideas."
These are important concepts: The purpose of the UI is to aid the user in thinking about what they are doing inside their machine. The UI should match what they are thinking about as closely as possible.
This is all well and good for users. It gives them pretty looking sites that are easy to use and understand. It leads to ideas about making user customization of content a certain flavor of easy that involves allowing for graphical interaction with the system.
But, the programming language is UI too.
I have recently graduated into a tier of programmers that write programs for other programmers to use. But, I have always known that unless you are building the system from nothing then you are the user of another person's UI. It may be called an API but it is still UI.
Programming languages are UI too, they are UI with a limited audience, developers. Just as EE folks design CPU with a "programmer's model" in mind so too should library developers develop software with application developers in mind.
The best way to do this is through "dog-fooding" which is to say that a developer should be forced to eat their own dog-food. When you develop something you should have to use it. If you can't use it you can't really make well informed decisions about how that product should work.
As a developer I try to get inside the minds of the people who use my products. I try to anticipate what they will do, how they will do it, and what ways they may want to do things differently. I can operate this way because after nearly two decades of working with computers in one fashion or another I am convinced there is no right or wrong intrinsic in the machines themselves.
Any meaning, purpose, or sense of working or brokenness is determined by the user. To make a system that is bug free you have to know the user. The technology therefore must be designed from the user back to the machine.
The problem is compounded by the fact that there are multiple types of users. You can build classifications for each of them profiling what their expectations are and building their view of the system around that. Working for one user is broken for another. To design a truly beautiful system you need to account for as many of your user classes as possible.
Often what system designers and developers leave out of their design consideration is themselves. Developers are users too. The must live with their own design choices over time and are prone to the same flaws that other user classifications are prone to as well.
The reasons technologies succeed and fail are tied up in the intersections of actual functionality and perceived function and value. These are emotional decisions as much as logical ones. Very technically speaking any Turing complete language is capable of any task that any other Turing complete language can do.
So the power we find in expressive software is actually the ability to cleanly express some thought or idea. We are attempting to communicate both to the machine and the user some expression of action, computation, art, or logic that will achieve the goal of the user. The software is most effective when the developer and the user understand each other through the lens of the software.
This is true for your web application, mobile application, programming language, or your developer API.
The web framework developer is working for their user which is the web application developer. The persistence layer developer is working for their user which is both the framework developer and the application developer. The application developer is working for the end users of their system.
The most effective developers will be the ones that realize this and deliberately cross as many disciplines as appropriate for their application space. This kind of renaissance developer will be most effective at knowing how what they will write will be received. This kind of developer will know implicitly that it is all about user interface and developers are users too.
Just as architects have known for generations that a building must be beautiful both on the outside and the inside yet still respect the laws of nature so too must computing grow up. So in my phrase "the power of software is the power of thoughts and ideas" is nestled the concept that not only power but beauty and value of software is the beauty and value of thoughts and ideas.
The metrics of our beauty in software is the cognitive dissonance we create in attempting to enable our users to create their art with our tools. When you create software you are implicitly directing the thoughts of another person or machine. Just as an Orwellian language could forbade certain thoughts your API could forbid a design or direct users down a path. The challenge is to match your design to the users' thoughts and direct them toward good solutions without imposing a mental fascism.
The converse is side of this is that you want to direct your users toward maintainable and workable designs. The goal should be to guide not to force. So working out an good design is a balancing act.
Just some thoughts that I've had while looking at possible improvements to an existing development framework.
Edit: Oh yes, I saw this shiny "follow" gadget and added it to the upper right hand corner. Won't you do me a favor and click it a few dozen times so I don't feel too bad about having "0" followers. Thanks.
2008-03-15
... with the Eclipse RCP
I've been working with Eclipse RCP and the SWT on a project lately.
I started writing software in C/C++ about a decade ago. I remember working with C programs and UI back then as state machines. Essentially the UI would set states that a processing thread would later act upon. If you think about it this was an early form of Model View Controller separation.
I understood this type of separation well because the main process and the UI process essentially communicated through a set of state variables. If you were smart you nailed down these state variables and kept them in a central location and made them easy to understand.
I actually worked this way for most of my career... except for a few years back around 2000 when I joined the Internet hordes... and except for the last 4 years when I started working on Ajax applications (well, we didn't call them Ajax applications until recently).
I was pretty well familiar with Ajax by the time an opportunity to work with SWT came around. I had one project in Swing in the fall of 2001 but since then Java UIs weren't in the picture for me. I'll tell you EJB3 was a welcome change from the J2EE work I had done back in 2001 but that's another post entirely. Let me focus on the SWT in this post.
When I was given the chance to take apart and enhance an Eclipse RCP project I saw it as a golden opportunity to gain familiarity again with Java desktop applications. That's not an area that I personally get to work with much. Most of my time is spent working with server-side and database projects... ironically my university emphasis was on 3D graphics in OpenGL... so a chance to work with graphics again was really welcome.
The SWT and Eclipse are not kind to new comers though. In particular you need to be careful of "kitchen sinking" your Eclipse IDE when you go to develop on it. The rule for how to keep Eclipse from getting "kitchen sunk" seems to be to keep the plugin base as minimal as possible. A bit of a pain especially since the Eclipse plugin manager will let you break Eclipse out-right by letting you get into a configuration that "does not contain the platform" (whatever that means) so that you have a nicely dead eclipse.
I hadn't realized that I had been lucky with eclipse up to this point. Most of my projects being server side didn't need advanced Eclipse features such as the GEF. Apparently if you try to use VE, GEF, jface, and eclipse.ui in the same project you're going to be doing some juggling.
Soon after I launched into this project I decided I needed training material since I wasn't intuitively picking up on SWT. The Visual Editor (VE) works with both SWT and Swing components but that's no consolation if you don't know either very well. It still isn't easy to make progress.
And this is where I think I'm starting to change my mind about a few things. I'm sure you've seen the GWT (that's the Google Web Toolkit if you didn't know) and it's basically a Swing/SWT type of web programming UI library. This is great if you like that kind of UI work... but really... I don't think I do.
I think I actually like Ajax and that mess of HTML and I think I like it because it's so easy to see what a UI element is going to look like. I think the HTML forces such a clean break between the controller and the view that it's actually beneficial. It's a disconcerting view for me to entertain because I actually come from a C/C++ background and I would probably prefer a more rigorous UI development environment.
In my utopia VRML would actually have lead somewhere instead of dying out. The concept behind a VRML interface and a web page is that presentation should be scripted. The idea of a mark up language hinting at a layout for a render engine is some what radical by itself but once you accept the idea ... you have to wonder why not use it everywhere?
I've actually had to write postscript drivers and code generators for postscript code. Postscript is actually a remarkable display oriented language. It is also far from a mark up language looking more like a custom graphics DSL than anything. So it's no shock to me that it may have taken two decades for the industry to accept the idea of markup for describing how to display things.
Consider that HTML and VRML are concerned with general descriptions of elements and leave the particulars of display up to the render engine. And this is the liberating feature of the markup language you get to care about only the particulars you want to. That frees the designer to paint with a broad brush and can separate the design and implementation functions very nicely.
Certainly the same discipline can be applied to work done with SWT, GWT, and Swing but that requires the effort and attention of the programmer. The web interface and its restrictions actually force the issue of the separation of design and function.
*sigh* but, design and function aren't actually separate are they... design dictates what a function must do and a function is only comprehended through the design of the interface to the function. A chair must have the shape of a chair... a cup must be a cup. Handles and legs and decoration can be added without too much troubling the core purpose of the artifact but ... really the design and function are one and the same.
So what am I saying about the SWT, GWT, Swing, or any other UI design API for that matter? I guess I'm trying to get at this idea that you need to be able to describe the UI in graphical terms. Component UI toolkits procedurally describe how a UI is built up in steps... and it's still procedural even if the procedures are encapsulated in objects.
Consider in all the UI toolkits I know (other than the Ajax ones) you build a menu object, build a menu item object, give the menu item object a code, enum, or pointer to a method to call, and then inject it into the menu object at menu initialization. This is very procedural and has virtually nothing to do with what a menu is graphically.
Now consider an Ajax design: You mark up a set of blocks to hold the menu items and apply style guides to them and then hook the onclick event to the appropriate action or link. The trick here is we are describing visual elements and the only thing that makes anything a menu is how it is used. That's a subtle but important paradigm shift.
It's subtle because a menu element and a menu object aren't conceptually that different. It's important because a menu element is really just a display block primitive with some style guides and a menu object is a much heavier complex and rich thing. But in the end they both do the same thing. It's just that one way is describing a thing that happens to get shown in a particular way graphically and the other is describing a graphical thing that happens to get used in a particular way programmatically.
The markups are interested in describing visual elements given a handful of visual primitives but the toolkits are focused on the aggregation of visual elements in a prescribed taxonomic hierarchy. That means that the toolkits are actually in danger of creating an impedance mismatch akin to the famous object-relational impedance mismatch that we've fought in OOP and RDBMS circles for the last decade or so.
I wonder if we won't see talk of a UI-Object impedence mismatch or other metaphorical terminology for the mismatch between UI and UX expressions and the creation of component based taxonomic object hierarchies that pervade the practice of OOP. This is an especially ironic argument to make since OOP was created to better match a graphical environment. It's actually almost silly to make the argument against using OOP for GUI work... but here I am entertaining the notion.
I am looking to technologies like ZUML in the Zk Framework to start showing us if markup-componentized hybrid frameworks are actually feasible in a non-proprietary system. For now it's simply not practical for a "main stream" application to entertain the use of a Zk based design. Unless you and your users are ready to surf the cutting edge you will have to ride along with the rest of us and our component based frameworks.
Or you could go with Ajax.
For now I've decided to continue slogging through my work with SWT and Eclipse but it's really because I've got too much into this particular direction to back out now. I will say I have high hopes for the GroovySWT project and I've already sampled the Groovy Swing project. These are very nifty projects that promise the creation of a builder or markup-like way to express GUIs in both Swing and SWT. Hopefully both projects can find ways to "macro" or simplify UI building in the process of providing a new way to rapidly build up UI. The SWT builder already provides a very intuitive container syntax that eliminates some of that counter-intuitive procedural buildup code you find in Eclipse RCP Perspective and Activator objects.
I for one welcome our new groovy-builder overlords.
I started writing software in C/C++ about a decade ago. I remember working with C programs and UI back then as state machines. Essentially the UI would set states that a processing thread would later act upon. If you think about it this was an early form of Model View Controller separation.
I understood this type of separation well because the main process and the UI process essentially communicated through a set of state variables. If you were smart you nailed down these state variables and kept them in a central location and made them easy to understand.
I actually worked this way for most of my career... except for a few years back around 2000 when I joined the Internet hordes... and except for the last 4 years when I started working on Ajax applications (well, we didn't call them Ajax applications until recently).
I was pretty well familiar with Ajax by the time an opportunity to work with SWT came around. I had one project in Swing in the fall of 2001 but since then Java UIs weren't in the picture for me. I'll tell you EJB3 was a welcome change from the J2EE work I had done back in 2001 but that's another post entirely. Let me focus on the SWT in this post.
When I was given the chance to take apart and enhance an Eclipse RCP project I saw it as a golden opportunity to gain familiarity again with Java desktop applications. That's not an area that I personally get to work with much. Most of my time is spent working with server-side and database projects... ironically my university emphasis was on 3D graphics in OpenGL... so a chance to work with graphics again was really welcome.
The SWT and Eclipse are not kind to new comers though. In particular you need to be careful of "kitchen sinking" your Eclipse IDE when you go to develop on it. The rule for how to keep Eclipse from getting "kitchen sunk" seems to be to keep the plugin base as minimal as possible. A bit of a pain especially since the Eclipse plugin manager will let you break Eclipse out-right by letting you get into a configuration that "does not contain the platform" (whatever that means) so that you have a nicely dead eclipse.
I hadn't realized that I had been lucky with eclipse up to this point. Most of my projects being server side didn't need advanced Eclipse features such as the GEF. Apparently if you try to use VE, GEF, jface, and eclipse.ui in the same project you're going to be doing some juggling.
Soon after I launched into this project I decided I needed training material since I wasn't intuitively picking up on SWT. The Visual Editor (VE) works with both SWT and Swing components but that's no consolation if you don't know either very well. It still isn't easy to make progress.
And this is where I think I'm starting to change my mind about a few things. I'm sure you've seen the GWT (that's the Google Web Toolkit if you didn't know) and it's basically a Swing/SWT type of web programming UI library. This is great if you like that kind of UI work... but really... I don't think I do.
I think I actually like Ajax and that mess of HTML and I think I like it because it's so easy to see what a UI element is going to look like. I think the HTML forces such a clean break between the controller and the view that it's actually beneficial. It's a disconcerting view for me to entertain because I actually come from a C/C++ background and I would probably prefer a more rigorous UI development environment.
In my utopia VRML would actually have lead somewhere instead of dying out. The concept behind a VRML interface and a web page is that presentation should be scripted. The idea of a mark up language hinting at a layout for a render engine is some what radical by itself but once you accept the idea ... you have to wonder why not use it everywhere?
I've actually had to write postscript drivers and code generators for postscript code. Postscript is actually a remarkable display oriented language. It is also far from a mark up language looking more like a custom graphics DSL than anything. So it's no shock to me that it may have taken two decades for the industry to accept the idea of markup for describing how to display things.
Consider that HTML and VRML are concerned with general descriptions of elements and leave the particulars of display up to the render engine. And this is the liberating feature of the markup language you get to care about only the particulars you want to. That frees the designer to paint with a broad brush and can separate the design and implementation functions very nicely.
Certainly the same discipline can be applied to work done with SWT, GWT, and Swing but that requires the effort and attention of the programmer. The web interface and its restrictions actually force the issue of the separation of design and function.
*sigh* but, design and function aren't actually separate are they... design dictates what a function must do and a function is only comprehended through the design of the interface to the function. A chair must have the shape of a chair... a cup must be a cup. Handles and legs and decoration can be added without too much troubling the core purpose of the artifact but ... really the design and function are one and the same.
So what am I saying about the SWT, GWT, Swing, or any other UI design API for that matter? I guess I'm trying to get at this idea that you need to be able to describe the UI in graphical terms. Component UI toolkits procedurally describe how a UI is built up in steps... and it's still procedural even if the procedures are encapsulated in objects.
Consider in all the UI toolkits I know (other than the Ajax ones) you build a menu object, build a menu item object, give the menu item object a code, enum, or pointer to a method to call, and then inject it into the menu object at menu initialization. This is very procedural and has virtually nothing to do with what a menu is graphically.
Now consider an Ajax design: You mark up a set of blocks to hold the menu items and apply style guides to them and then hook the onclick event to the appropriate action or link. The trick here is we are describing visual elements and the only thing that makes anything a menu is how it is used. That's a subtle but important paradigm shift.
It's subtle because a menu element and a menu object aren't conceptually that different. It's important because a menu element is really just a display block primitive with some style guides and a menu object is a much heavier complex and rich thing. But in the end they both do the same thing. It's just that one way is describing a thing that happens to get shown in a particular way graphically and the other is describing a graphical thing that happens to get used in a particular way programmatically.
The markups are interested in describing visual elements given a handful of visual primitives but the toolkits are focused on the aggregation of visual elements in a prescribed taxonomic hierarchy. That means that the toolkits are actually in danger of creating an impedance mismatch akin to the famous object-relational impedance mismatch that we've fought in OOP and RDBMS circles for the last decade or so.
I wonder if we won't see talk of a UI-Object impedence mismatch or other metaphorical terminology for the mismatch between UI and UX expressions and the creation of component based taxonomic object hierarchies that pervade the practice of OOP. This is an especially ironic argument to make since OOP was created to better match a graphical environment. It's actually almost silly to make the argument against using OOP for GUI work... but here I am entertaining the notion.
I am looking to technologies like ZUML in the Zk Framework to start showing us if markup-componentized hybrid frameworks are actually feasible in a non-proprietary system. For now it's simply not practical for a "main stream" application to entertain the use of a Zk based design. Unless you and your users are ready to surf the cutting edge you will have to ride along with the rest of us and our component based frameworks.
Or you could go with Ajax.
For now I've decided to continue slogging through my work with SWT and Eclipse but it's really because I've got too much into this particular direction to back out now. I will say I have high hopes for the GroovySWT project and I've already sampled the Groovy Swing project. These are very nifty projects that promise the creation of a builder or markup-like way to express GUIs in both Swing and SWT. Hopefully both projects can find ways to "macro" or simplify UI building in the process of providing a new way to rapidly build up UI. The SWT builder already provides a very intuitive container syntax that eliminates some of that counter-intuitive procedural buildup code you find in Eclipse RCP Perspective and Activator objects.
I for one welcome our new groovy-builder overlords.
2007-09-01
Grails Widget list...
Take a look at this...
... which I got from the Grails site. So, if you want to use the code generators to create views for a form and want a string to have a text area as its input you can use the constraints on the object to specify text area. But wait, what other widgets can you specify?
I did a little digging in the Grails source code and came up with this list....
def constraints = {
description(widget:'textarea')
}
... which I got from the Grails site. So, if you want to use the code generators to create views for a form and want a string to have a text area as its input you can use the constraints on the object to specify text area. But wait, what other widgets can you specify?
I did a little digging in the Grails source code and came up with this list....
- textField
- hiddenField
- submitButton
- field
- textArea
- form
- actionSubmit
- actionSubmitImage
- datePicker
- renderNoSelectionOption
- timeZoneSelect
- localeSelect
- currencySelect
- select
- checkBox
- radio
2006-12-04
Full Body User Interface
One of my graduate school professors runs the Body Language User Interface project called BLUI for short. It is a unique Human Computer Interaction (HCI) project that allows a user to paint in three dimensions. Projects like BLUI probably inspired the current generation of console controllers on products like the PlayStation3 and the Wii.
The prolonged sedentary use of computers is simply unhealthy. Human bodies were not designed to sit immobile in front of a screen for hours on end. Physical exercise may be important for mental health in general and is completely absent from conventional human-computer interactions.
The computer is arguably the most profound advance of human technology since the advent of the written word. Many of us today live computer-life-styles that would be impossible without these devices. Yet they are slowly killing us.
Wii users are now talking about soreness and fatigue from extended use of the Wii-mote. Repetitive-motion injuries are obviously a concern with any UI and the Wii offers the opportunity for you to experience tennis-elbow caused by a video game. These are just the hazards of owning a body and playing a sport.
And, now we can start to call some video games sports. Perhaps Electronic Sports are on the horizon? All the better. It means that our computers are going to stop being shackles that tie us to the desktop.
Some distant future may even give us a Full Body Interface (FuBI) where the users will dance to communicate with their machines. Dances that control fighter jets. Dances that control robot probes on the surface of Mars. Dances that communicate how to conduct the surgical removal of a tumor.
It's just an idea.
The prolonged sedentary use of computers is simply unhealthy. Human bodies were not designed to sit immobile in front of a screen for hours on end. Physical exercise may be important for mental health in general and is completely absent from conventional human-computer interactions.
The computer is arguably the most profound advance of human technology since the advent of the written word. Many of us today live computer-life-styles that would be impossible without these devices. Yet they are slowly killing us.
Wii users are now talking about soreness and fatigue from extended use of the Wii-mote. Repetitive-motion injuries are obviously a concern with any UI and the Wii offers the opportunity for you to experience tennis-elbow caused by a video game. These are just the hazards of owning a body and playing a sport.
And, now we can start to call some video games sports. Perhaps Electronic Sports are on the horizon? All the better. It means that our computers are going to stop being shackles that tie us to the desktop.
Some distant future may even give us a Full Body Interface (FuBI) where the users will dance to communicate with their machines. Dances that control fighter jets. Dances that control robot probes on the surface of Mars. Dances that communicate how to conduct the surgical removal of a tumor.
It's just an idea.
Subscribe to:
Posts (Atom)