Archive for September 2013

Scrolling Sites on iOS – mobile apps requires something other than HTML/CSS

UPDATE: My fix for this issue was using webkit’s translate3D transform to sidestep the performance issues (for the most part). Here’s a nice performance comparison by Paul Irish. There’s 3 hacks to make it work:

1) you have to use translate3D, not translate2D, for the iPad’s hardware acelleration to work.

2) You have to turn -webkit-backface-visibility to hidden if your elements are larger than the screen

3) You want to set up a default -webkit-transform: translate3d(0,0,0); inside your CSS to prevent a jerk on first touch.

Here’s my code that implements this, while unfortunately disabling momentum for scrolls.


HTML/CSS is arguably the most successful domain specific language we have – it makes it dead simple to write applications containing graphical elements. I claim that it’s an instance of using a high level language (CSS and HTML) to get high performance programs (web apps) by embedding domain specific knowledge into the runtime (document layout, text rendering, interaction events).

Now consider the issues of doing a scrolling narrative with pinned elements & parallax scrolling.

The HTML/CSS abstraction is not a good fit for the layout & positioning requirements of this, which I will try to demonstrate. As you scroll along, you have imperative JS code tracking your scroll, and this fires off changes to the declarative CSS language.

Part of the annoyance is the difference in expression – three languages involved in this process – and the fact that crossing these barriers have performance implications, especially on mobile (this claim is worth testing)

On mobile, here’s the event process:
- the user starts scrolling (1)
- all JS execution is paused
- the user stops scrolling
- JS execution is resumed, receives a scroll event, checks where the page are
- if the page has scrolled past a predetermined point (this in itself is hairy to calculate) we now change the css for a div from position:relative to position:fixed (2)
- the css takes a moment to reconfigure and redraw the display
- the image jumps to a new location since the screen has moved in the meantime
- now as we scroll, the image stays at a fixed location on screen, moving relative to the document (3)
- the user keeps scrolling, past the bottom of the element’s pinned region.
- the css for the div is now updated to a new position

Naturally a big issue here comes in with iOS pausing JS execution, but even without that you get shuddery twitchy behavior, because the high level API that is supposed to give you higher performance since it embeds domain specific information into the runtime is being misused here since it has no way of expressing these constraints.

Potential solutions

Going off the observation that we cannot express these constraints in CSS, perhaps we should be able to express a constraint-based CSS rule for this. Something along the lines of “position of image div is equal to top of text if viewport top is higher, or top of viewport, or bottom of text if viewport is lower”.

This has several concerns for a constraint-based approach: these are not linear or quadratic constraints, they are turned on and off. That is, as the location of the viewport changes, the constraint on the image div changes – initially it is in line with the top of the text, then it becomes equal to the position of the viewport on the document, then it becomes equal to the bottom of the text block.

This means we would have to construct a function as the solution to the constraint function, given the current viewport location, would determine the locations of elements. I believe this would be challenging to do well, and warrants further exploration. This is also the same problem that comes up in automatic document layout using templates! People like Greg Badros and Alan Borning’s Constraint Cascading Style Sheets for the Web, and the work done on Adaptive Grid-Based Document Layout had to solve similar problems (either through prioritizing different constraints or by discretizing the problem and using a big offline dynamic-programing solver). Perhaps someone has addressed this issue in the constraint literature already, and it’s just waiting for us to use it?

A different direction of thinking would be to get rid of the declarative CSS/HTML manner of expressing this. What would be a more appropriate low-level approach? We can find inspiration here in thinking about how this same behavior would be implemented in a native iOS app. For the moment I’d have to leave this for future pondering.

Conclusion

I’m becoming more convinced that, rather than piling features into HTML and CSS, we can come up with a lower-level abstraction that has many of the same benefits (easily expressible graphical applications) but with an API better matched to the applications we’re building rather than static document layout. The declarative nature of CSS makes it more challenging to build this as an extensible system, but I think the gap between “mobile webapps” and “native apps” can be shrunk if we reimagine CSS. It’ll make my work a lot easier!