Fix mismatch screen width and height and buffer synchronization when using Linux framebuffer backend

[a1091150]

The commit 980bd4c supports Linux framebuffer backend, but the behavior is weird on my machine.

Typical laptop with an intel processor.
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          39 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   8
  On-line CPU(s) list:    0-7
Vendor ID:                GenuineIntel
  Model name:             12th Gen Intel(R) Core(TM) i3-1215U

The root case is that some hardware require a cache synchronization when using the framebuffer.

Problem:

1. Mismatch width and height and 3 windows are drawn(in the screenshot).
2. Some updates (use twin_screen_damage to mark the field to be updated) are not apply immediately on computer screen.

One possible fix is in a1091150/mado@fbe9e00, the code is copied from FOSDEM 2020 - Back to the Linux Framebuffer! and sync_cache.c

After the below commit, another problem appeared is that the whole screen is not rendered when startup. To fix this, simply use twin_screen_damage to mark the whole scrreen to be damaged, shown in the following code(a1091150/mado@977463a).

static bool twin_fbdev_work(void *closure)
{
    twin_screen_t *screen = SCREEN(closure);
    static bool run_once = true;
    if (run_once) {
        run_once = false;
        twin_screen_damage(screen, 0, 0, screen->width, screen->height);
    }

    if (twin_screen_damaged(screen))
        twin_screen_update(screen);
//...
}

[shengwen-tw]

@a1091150 :

Hi, I can help fixing the problem but it is a little harder for me to duplicate the problem.
Are you interested in solving this problem? Otherwise I can take over it.

[a1091150]

@a1091150 :

Hi, I can help fixing the problem but it is a little harder for me to duplicate the problem. Are you interested in solving this problem? Otherwise I can take over it.

Yes, I am interested in this problem, but I am not familiar with Linux framebuffer. This will take a while.
Currently I tested it on virtual machine (VirtualBox) and it is more broken now.

[jserv]

Consider to call twin_screen_register_damaged after the call of twin_set_work. That is,

    twin_set_work(twin_fbdev_work, TWIN_WORK_REDISPLAY, ctx);

    /* Enable immediate refresh */
    twin_screen_register_damaged(ctx->screen, twin_fbdev_damaged, ctx);
    ...

And the callback twin_fbdev_damaged is shown below:

static void twin_fbdev_damaged(void *closure)                                                                                                                 
{
    twin_screen_t *screen = SCREEN(closure);
    twin_fbdev_t *tx = PRIV(closure);

    if (tx->vt_active /* VT switch is ready */ && twin_screen_damaged(screen))
        twin_screen_update(screen);
}

However, the current fbdev backend lacks of VT switch operations. That is, consider the following:

static void twin_fbdev_switch(twin_fbdev_t *tf, int activate)
{
    tx->vt_active = activate;
    
    /* Upon activation */
    if (activate) {
        /* Switch complete */
        ioctl(tx->vt_fd, VT_RELDISP, VT_ACKACQ);

        /* Restore fbdev settings */
        if (twin_fbdev_apply_config(tf)) {
            tx->vt_active = true;

            /* Mark entire screen for refresh */
            if (tx->screen)
                twin_screen_damage(tx->screen, 0, 0, tx->screen->width,
                                   tx->screen->height);
        }
    } else {
        /* FIXME: should expose some option to disallow them */
        ioctl(tx->vt_fd, VT_RELDISP, 1);

        tx->vt_active = 0;

        if (tx->fb_base != MAP_FAILED)
            munmap(tx->fb_base, tx->fb_len);
        tx->fb_base = MAP_FAILED;
    }
}

static bool vt_switch_pending;

static bool twin_fbdev_work(void *closure)
{
    twin_fbdev_t *tx = PRIV(closure);

    if (vt_switch_pending) {
        twin_fbdev_switch(tf, !tx->vt_active);
        vt_switch_pending = false;
    }

    return true;
}

static void twin_fbdev_vtswitch(int sig)
{
    signal(sig, twin_fbdev_vtswitch);
    vt_switch_pending = true;
}

static bool twin_fbdev_setup_vt(twin_fbdev_t *tx, int switch_sig)
{
    struct vt_mode vtm;
    struct termios tio;

    if (ioctl(tx->vt_fd, VT_GETMODE, &vtm) < 0) {
        SERROR("can't get VT mode");
        return 0;
    }
    vtm.mode = VT_PROCESS;
    vtm.relsig = switch_sig;
    vtm.acqsig = switch_sig;

    signal(switch_sig, twin_fbdev_vtswitch);
    tx->vt_swsig = switch_sig;

    if (ioctl(tx->vt_fd, VT_SETMODE, &vtm) < 0) {
        SERROR("can't set VT mode");
        signal(switch_sig, SIG_IGN);
        return 0;
    }

    tcgetattr(tx->vt_fd, &tx->old_tio);

    ioctl(tx->vt_fd, KDGKBMODE, &tx->old_kbmode);
    ioctl(tx->vt_fd, KDSKBMODE, K_MEDIUMRAW);

    tio = tx->old_tio;
    tio.c_iflag = (IGNPAR | IGNBRK) & (~PARMRK) & (~ISTRIP);
    tio.c_oflag = 0;
    tio.c_cflag = CREAD | CS8;
    tio.c_lflag = 0;
    tio.c_cc[VTIME] = 0;
    tio.c_cc[VMIN] = 1;
    cfsetispeed(&tio, 9600);
    cfsetospeed(&tio, 9600);
    tcsetattr(tx->vt_fd, TCSANOW, &tio);

    ioctl(tx->vx_fd, KDSETMODE, KD_GRAPHICS);

    return true;
}

Check the following implementations:

• LVGL: src/drivers/display/fb/lv_linux_fbdev.h, src/drivers/display/fb/lv_linux_fbdev.c
• AWTK: awtk-linux-fb is more complete.

[a1091150]

@a1091150 :
Hi, I can help fixing the problem but it is a little harder for me to duplicate the problem. Are you interested in solving this problem? Otherwise I can take over it.

Yes, I am interested in this problem, but I am not familiar with Linux framebuffer. This will take a while. Currently I tested it on virtual machine (VirtualBox) and it is more broken now.

Another problem I found is visible resolution and virtual resolution are different on Virtualbox, and miscalculates the memory region to be rendered. Use line_length in struct fb_fix_screeninfo to replace these resolution and solve the miscalculation.

Run sudo fbset - i gives the following information, it reads and prints out from struct fb_fix_screeninfo:

mode "800x600"
    geometry 800 600 2048 2048 32
endmode

Frame buffer device information:
    ......
    LineLength  : 8192
    Accelerator : No

On my laptop is gives:

mode "1920x1080"
    geometry 1920 1080 1920 1080 32
    ....
endmode
Frame buffer device information:
    ....
    LineLength  : 7680
    Accelerator : No

Solution:
Use tx->fb_fix.line_length to replace screen->width.

static void _twin_fbdev_put_span(twin_coord_t left,
                                 twin_coord_t top,
                                 twin_coord_t right,
                                 twin_argb32_t *pixels,
                                 void *closure)
{
    twin_screen_t *screen = SCREEN(closure);
    twin_fbdev_t *tx = PRIV(closure);

    if (tx->fb_base == MAP_FAILED)
        return;

    twin_coord_t width = right - left;
-    off_t off = top * screen->width + left;
-    uint32_t *dest =
-        (uint32_t *) ((uintptr_t) tx->fb_base + (off * sizeof(uint32_t)));
+    unsigned char *dest =
+        (unsigned char *) tx->fb_base + tx->fb_fix.line_length * top + 4 * left;
    memcpy(dest, pixels, width * sizeof(uint32_t));
}

[jserv]

Another problem I found is visible resolution and virtual resolution are different on Virtualbox, and miscalculates the memory region to be rendered. Use line_length in struct fb_fix_screeninfo to replace these resolution and solve the miscalculation.
[..]
Solution: Use tx->fb_fix.line_length to replace screen->width.

Can you submit a pull request for testing and reviewing purpose?

[a1091150]

Can you submit a pull request for testing and reviewing purpose?

There are two test cases and are being tested on my Ubuntu laptop and VirtualBox.

1. Start demo-fbdev in a pseudoterminal, and switch to another virtual terminal via ctrl+alt+F3 to tty3.
2. First switch to tty3, and start demo-fbdev.

Case 2 works fine.
Case 1 will give the result as below screenshot. Some render is overwritten on the display, and cursor's movement looks lagging. Note that I can still type some words in tty3.

In case 1, I find out another way to make it not lagging is updating the cache, the code below is from sys_cache.c, but I am not sure whether it is an appropriate way.

#define FBIO_CACHE_SYNC 0x4630
unsigned int args[2];
args[0] = tx->fb_base;
args[1] = tx->fb_base + tx->fb_var.yres_virtual * tx->fb_fix.line_length;
ioctl(tx->fb_fd, FBIO_CACHE_SYNC, args);

mado with framebuffer can start from different terminals like tty and pts (pseudoterminal), and the code 980bd4c assumes the terminal and the target framebuffer is on the same display, #62 (comment) can apply onto the case 2. However this can not work on the case 1 that framebuffer and terminal are not on the same display.

The pull request: #65
If the origin design aims to case 2, then the pull request will only fix miscalculated memory region in _twin_fbdev_put_span , and cache updating code will not be pulled.

static void _twin_fbdev_put_span(twin_coord_t left,
                                 twin_coord_t top,
                                 twin_coord_t right,
                                 twin_argb32_t *pixels,
                                 void *closure)
{
    twin_screen_t *screen = SCREEN(closure);
    twin_fbdev_t *tx = PRIV(closure);

    if (tx->fb_base == MAP_FAILED)
        return;

    twin_coord_t width = right - left;
-    off_t off = top * screen->width + left;
-    uint32_t *dest =
-        (uint32_t *) ((uintptr_t) tx->fb_base + (off * sizeof(uint32_t)));
+    off_t off = tx->fb_fix.line_length * top + 4 * left;
+    unsigned char *dest = (unsigned char *) tx->fb_base + off;
    memcpy(dest, pixels, width * sizeof(uint32_t));
}

[jserv]

@huaxinliao, Please install VirtualBox as @a1091150 suggested, and see if the above issue still occurs in the latest main branch.

[huaxinliao]

In Case 2, when I attempted to gradually move the cursor to the right by shifting my finger to the right, I observed that the cursor on the screen moved to the right in fixed increments rather than in a smooth, continuous motion. At this time, the cursor was unable to move upward or in any other direction.

Case2.mp4

[huaxinliao]

I attempted to reproduce Case 1. However, after launching demo-fbdev, I tried switching to tty3 using Ctrl+Alt+F3, but it did not work. @a1091150 Could you please help me confirm this?

Case1.mp4

[a1091150]

In Case 2, when I attempted to gradually move the cursor to the right by shifting my finger to the right, I observed that the cursor on the screen moved to the right in fixed increments rather than in a smooth, continuous motion. At this time, the cursor was unable to move upward or in any other direction.

Case2.mp4

Same happened on Virtualbox with Ubuntu 22.04 as a guest OS and Windows as a host OS. No same problem on a real hardware.

I attempted to reproduce Case 1. However, after launching demo-fbdev, I tried switching to tty3 using Ctrl+Alt+F3, but it did not work. @a1091150 Could you please help me confirm this?

Case1.mp4

Start a terminal (i.e. by clicking a terminal icon in gnome desktop). In this terminal, run sudo ./demo-fbdev, then switch to virtual terminal (i.e. by typing ctrl + alt + f3 to tty3 and ctrl + alt + f4 to tty4). This test case works fine.

Originally, this issue wants to fix framebuffer region which cause #62 (comment) and the screenshot presented. The lastest main branch fixs this and supports more different bits of a pixel.

[huaxinliao]

I conducted the experiment based on @a1091150's suggestion. During the process, I discovered that the framebuffer backend simultaneously exists on multiple TTYs. Initially, only part of the window is displayed on the screen. However, when I move the cursor within the window area, the full window becomes visible on the screen.

Case_1.mp4

[jserv]

During the process, I discovered that the framebuffer backend simultaneously exists on multiple TTYs. Initially, only part of the window is displayed on the screen.

You can check how DirectFB2 deals with virtual terminal. See

• systems/fbdev/fbdev_vt.h
• systems/fbdev/fbdev_vt.c
• systems/fbdev/fbdev_system.c