This article delves into how the React component HighTable overcomes browser limitations with five core technologies to achieve a virtual scrolling solution that smoothly displays and operates billions of rows of data on the web.
Summary
The article details the vertical scrolling technology stack employed by the HighTable component when handling massive datasets (billions of rows). The core challenges lie in browser memory limitations, DOM rendering pressure, and the maximum height limit of native HTML elements (e.g., Firefox's 17 million pixels). The author proposes five key technologies: 1. Lazy loading data frames, fetching visible rows on demand; 2. Table slicing, rendering only DOM nodes within the viewport; 3. Infinite pixel downsampling, breaking through browser height limits via mathematical mapping; 4. Pixel-level precise scrolling, compensating for precision loss due to downsampling using a "local + global" dual-mode logic; 5. Two-step random access, enabling keyboard navigation and programmatic jumps for extreme data volumes. This solution is entirely based on native HTML elements, requires no Canvas rendering, and balances performance with accessibility.
Main Points
* 1. Implement on-demand lazy loading and caching mechanisms through DataFrames.Design asynchronous fetch and synchronous getCell interfaces to load only the cell data required for the current viewport and cache it in memory, preventing massive data from overwhelming browser memory.
* 2. Utilize table slicing technology to maintain a constant number of DOM nodes.Introduce a canvas layer between the viewport and the table, rendering only about 30 visible rows of data, ensuring rendering overhead remains low regardless of the total data volume.
* 3. Introduce a downscaleFactor to break through browser height limitations.Addressing the browser's maximum height limit of approximately 17 million pixels, when the data volume is too large, the scrollbar displacement is proportionally mapped to a larger virtual space, enabling infinite row navigation.
* 4. Design a "local + global" dual-mode scrolling logic to solve the precision loss problem.By judging the scroll increment size, it automatically switches between global jumps and local pixel offsets, ensuring users can both quickly navigate and perform fine-grained row-by-row scrolling.
* 5. Support complex keyboard interactions through two-step random access and scroll decoupling.Separate vertical and horizontal scrolling logic, use flags to control the execution order of programmatic scrolling, ensuring pixel-level cell focusing can still be achieved across billions of rows of data.
Metadata
AI Score
91
Website mp.weixin.qq.com
Published At Today
Length 7884 words (about 32 min)
Sign in to use highlight and note-taking features for a better reading experience. Sign in now
This article delves into how the React component HighTable overcomes browser limitations with five core technologies to achieve a virtual scrolling solution that smoothly displays and operates billions of rows of data on the web.
Summary
The article details the vertical scrolling technology stack employed by the HighTable component when handling massive datasets (billions of rows). The core challenges lie in browser memory limitations, DOM rendering pressure, and the maximum height limit of native HTML elements (e.g., Firefox's 17 million pixels). The author proposes five key technologies: 1. Lazy loading data frames, fetching visible rows on demand; 2. Table slicing, rendering only DOM nodes within the viewport; 3. Infinite pixel downsampling, breaking through browser height limits via mathematical mapping; 4. Pixel-level precise scrolling, compensating for precision loss due to downsampling using a "local + global" dual-mode logic; 5. Two-step random access, enabling keyboard navigation and programmatic jumps for extreme data volumes. This solution is entirely based on native HTML elements, requires no Canvas rendering, and balances performance with accessibility.
Main Points
* 1. Implement on-demand lazy loading and caching mechanisms through DataFrames.
Design asynchronous fetch and synchronous getCell interfaces to load only the cell data required for the current viewport and cache it in memory, preventing massive data from overwhelming browser memory.
* 2. Utilize table slicing technology to maintain a constant number of DOM nodes.
Introduce a canvas layer between the viewport and the table, rendering only about 30 visible rows of data, ensuring rendering overhead remains low regardless of the total data volume.
* 3. Introduce a downscaleFactor to break through browser height limitations.
Addressing the browser's maximum height limit of approximately 17 million pixels, when the data volume is too large, the scrollbar displacement is proportionally mapped to a larger virtual space, enabling infinite row navigation.
* 4. Design a "local + global" dual-mode scrolling logic to solve the precision loss problem.
By judging the scroll increment size, it automatically switches between global jumps and local pixel offsets, ensuring users can both quickly navigate and perform fine-grained row-by-row scrolling.
* 5. Support complex keyboard interactions through two-step random access and scroll decoupling.
Separate vertical and horizontal scrolling logic, use flags to control the execution order of programmatic scrolling, ensuring pixel-level cell focusing can still be achieved across billions of rows of data.
Key Quotes
* Solutions that work well at a small scale become overwhelmed once data volume escalates.
* Because canvas height grows linearly with the number of rows... it can render at most 500,000 rows. Our solution in HighTable is to downsample the scrollbar's resolution.
* Local scrolling means moving the table slice pixel by pixel, while global scrolling means jumping directly to the position indicated by the scrollbar.
* No need to fake scrollbars, no need to render tables with canvas elements — we rely solely on the Web platform itself.
* This solution is effective when the full table height does not exceed the square of the maximum canvas height... guaranteeing 1px precision for up to 2 trillion rows of data.