[css-view-transitions-1] Define behaviour when capturing image for a large element. #8561
Comments
|
Summary for CSSWG: Problem:
Proposal:
Questions:
|
This was a bit confusing for me since the content of the old/new element is also called a view. :) I think you meant to say: "capture the area currently visible"?
Did you mean what should the natural dimensions of the old/new images be in these cases? Didn't follow what "view" was referring to here.
Noting from offline discussion, it can be via |
|
The CSS Working Group just discussed
The full IRC log of that discussion<emeyer> JakeA: When we capture the content of an old view, we don’t know how it might be animated yet<emeyer> …Part of the element might be out of view, and could come in view as part of the transition <emeyer> …Some elements are very large and we can’t feasibly capture all of it <emeyer> …Image transitioning the body element <emeyer> …Proposal is, we’ll base the dimensions of the transition group on the element, but say that browsers don’t have to capture every pixel, while suggesting they capture beyond the viewport <emeyer> …If possible/performant to do so <emeyer> …Are we okay to handwave like that, so we let UAs act as they see best? <TabAtkins> q+ <emeyer> …Further, what should the natural width and height of the in- and out-states be when a browser chooses not to capture every pixels? <emeyer> …Also, should those things be script-visible? <khush> q+ <emeyer> TabAtkins: I think we should require at least the visible area and some amount outside is good, maybe with a minimum explicit margin like 50% beyond <Rossen_> ack TabAtkins <emeyer> …We should expose the size of the thing being captured <emeyer> fantasai: +1 to Tab, plus the size should be the size it actually is, even if there isn’t painting inside the entire range <flackr> +1 <emeyer> …We should require or very strongly recommend capturing beyond the viewport as well <Rossen_> ack fantasai <emeyer> JakeA: So if there’s a case with many elements layered on top of each other, they’re all in the viewport; if device can’t handle all that, should we just skip the transition? <emeyer> …I think that’s a good general case, that if a UA doesn’t feel it can handle a given transition, it should skip the transition <fantasai> s/as well/as well, probably +1 viewport in each direction as the minimum range to capture/ <emeyer> khush: +1 to fantasai, which is kind of what I was implementing anyway, so I’m okay with the spec recommending taking the root as a barometer for how much to expand beyond the visible viewport <emeyer> …The spec says the natural size of the image is equal to what you capture and everything around it is transparent <emeyer> …object view box exposes how much the UA decided to paint <chrishtr> q+ <emeyer> fantasai: I don’t think you should use object view box to set this, even if you use the same computation internally <emeyer> khush: So the computed value is meant to be ink-overflow-rectangle <Rossen_> ack khush <emeyer> chrishtr: We would just not raster things we can’t put into memory, so developers can’t observe anything about this behavior <emeyer> fantasai: Exactly <flackr> +1 <vmpstr> q? <emeyer> JakeA: We can say the exposed value doesn’t say anything about optimizations <emeyer> vmpstr: I think the object view box is the problem, because we need to make it the same as ink overflow, but if that’s infinite\ <emeyer> flackr: It’s the same as if you hadn’t rendered everything <emeyer> khush: In the spec it says to paint everything in the ink overflow rectangle <emeyer> …We pretend an image is the size of the ink overflow rect <emeyer> chrishtr: A dev wouldn’t know whether we painted the whole thing or not, only the user can tell <khush> q+ <Rossen_> ack khush <emeyer> khush: Base case where the UA can raster the whole thing, and element has drop shadow, so ink overflow is bigger than object box <chrishtr> q+ <emeyer> …UA should compute a view box it applies to the element such that when you render this, the boxes should coincide at the same origin <emeyer> …If the UA hasn’t painted the whole thing, and the dev msses with the object view box, the effect is the same as if the whole thing was painted <emeyer> …My question is, to implement this, can we not let devs change object view box? <emeyer> …When they read it, we can give the the value the spec wants; also the device specs wouldn’t be exposed? <flackr> q+ <emeyer> fantasai: What is the ink overflow rectangle of something that has a box shadow? <emeyer> chrishtr: There’s spec language about this <emeyer> fantasai: Do we want to expose this? <flackr> q- <emeyer> khush: You could call getcomputedstyle and don’t see anything, which would make implementation easier <emeyer> fantasai: That seems better, and if the dev wants to manipulate they don’t have to know about the internals <emeyer> chrishtr: We could take this and whether we should add a new !important to a new rule <emeyer> fantasai: I think we agree the rasterization is not exposed to devs but the returned values are spec-consistent <Rossen_> ack chrishtr <fantasai> proposal: UA can limit rasterization for perf limitations, but must size the element as if it was fully rasterized <emeyer> chrishtr: Agreed <emeyer> JakeA: I think we generally agreed the spec should suggest an overflow amount, was it one viewport in each direction? <fantasai> proposal: rasterization should cover at least the visible area of the viewport + one viewport in each direction <emeyer> JakeA: Also in out-of-memory cases the transition is skipped <fantasai> proposal: if the UA cannot performantly perform the view transition (for any reason) it must skip the transition <emeyer> TabAtkins: We don’t usually specify memory problem recovery because it can show up whenever <emeyer> fantasai: I think we have three proposed resolutions <emeyer> …One (see above) <TabAtkins> Exception is if OOM actually causes a security issue. In all other cases OOM behavior is explicitly undefined. <emeyer> …Two look one viewport in each direction for overflow handling <emeyer> …Three is the error fallback is to skip the transition <khush> q+ <emeyer> …And that you never do half a transition, you either do all or none <emeyer> khush: One viewport in each direction should be a recommendation rather than a requirement <emeyer> …Only requirement is that what’s visible must be captured <fantasai> proposal: UA can limit rasterization for perf limitations, but must size the element as if it was fully rasterized <fantasai> proposal: if the UA cannot performantly perform the view transition (for any reason) it must skip the transition <fantasai> proposal: rasterization should cover at least the visible area of the viewport + one viewport in each direction <emeyer> RESOLVED: user agent can limit rasterization for performance limitations, but must size the element as if it was fully rasterized <JakeA> proposal: Must: rasterization should cover at least the visible area of the viewport Should: + one viewport in each direction <JakeA> proposal: Must: rasterization should cover at least the visible area of the viewport <flackr> must + should = ? <JakeA> proposal: Must: rasterization covers at least the visible area of the viewport <emeyer> RESOLVED: rasterization must cover at least the visible area of the viewport <emeyer> JakeA: We need to consider the directions and how much overflow SHOULD be captured async <flackr> q+ <Rossen_> ack khush <bkardell_> I think no <emeyer> flackr: Skipping the transition is dev-visible so we’re exposing device capabilities <emeyer> fantasai: You can limit rasterization but if you’re getting to the point you can’t even transition what’s on-screen, you should skip the whole thing <bkardell_> q+ <Rossen_> ack bkardell_ <emeyer> flackr: We could not paint the transition but still run it so the dev knows it happened <emeyer> bkardell_: You’re saying skipping the transition is dev-visible but is it? Because a device can just not support that or animations can be turned off with prefers-reduced-motion <emeyer> Jake:We’ve decided those can be exposed, but giving away GPU memory is a fingerprint <emeyer> bkardell_: I see <emeyer> khush: Is it safer to say that this should silently fail by not painting things? <emeyer> …We can already hit this with filters and blurs and such and that’s silent in Chrome <emeyer> JakeA: I think we have to go back and come up with a plan here <emeyer> Rossen: So we’ll have to postpone this resolution |
|
@khushalsagar @atanassov the end of the meeting here is: So I don't think this is resolved yet? Perhaps we should wait with defining this until we resolve the issue around exposing hardware limitations? |
|
A note about the previous discussion: it is assumed that the in-viewport rect + buffer is always small enough to be captured. This doesn't take some cases into account. Think of a 500x10000 DIV with transform: rotateX(85deg). It would have most or all of its pre-transformed 5m pixels "in viewport". IMO the direction of the discussion was good, we should mention that the UA might not capture all the pixels due to hardware constraints, but that those constraints wouldn't affect the observable flow of the transition. |
|
@noamr, the resolution we had on this issue was captured in the comment here.
I didn't follow this. Was trying the test case here but I'm likely misunderstanding what you said. This does bring up a good point that when we say "visible area in the viewport", we mean an intersection between the snapshot root and the element's quad based on its screen space transform. The transform for that is already defined in the spec here. The end of the discussion was about a different point: Assuming we do have to capture everything in the viewport to be visually correct, but we don't have enough GPU memory to do it, should we abort the transition? And if so, should that failure be visible to authors or silent. This is a rare edge case that can be discussed as a follow up. But it doesn't alter the resolution that the UA must capture what's in the visible viewport and any extra buffer beyond that is a UA detail. |
|
Got it. I thought the phrase "So we’ll have to postpone this resolution" meant the whole resolution. Thanks for clarifying. |
A simpler case: a DIV with |
|
Ah, ok. In this case we can downscale the element's painting so the rasterized output fits within the hardware constraints. The default rendering (outside of this feature) would already do that, elements are rasterized based on their final scale in the viewport. Your example brings up a good point though that while the spec mandates rasterizing the visible viewport, it doesn't mandate a rasterization quality. This is reasonable. FWIW, a similar CSS feature also doesn't mandate rasterization quality here, but also renders the whole element in the image for this case. |
|
OK. I'll prepare a PR based on the resolution, stating that rasterizing the visible viewport is mandated but not in a particular quality. |
The default behaviour for View Transitions when capturing the image of an element is to paint its entire ink overflow rectangle. But there are cases where this is not possible. A common one is the element exceeding max texture size supported by the hardware on that device. It can technically be done (by tiling the image) but is complicated. It's also likely we'll hit cases where painting the whole element will have significant memory overhead and clipping the snapshot makes sense.
We want the snapshot to include at least the intersection of the element's ink overflow rectangle and the snapshot root (a.k.a the visible viewport). Other than that, it's unclear how prescriptive the spec needs to be here.
This issue is tracked here in the spec.
The text was updated successfully, but these errors were encountered: