1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
|
<?xml version='1.0' encoding='UTF-8'?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>Fryzek Concepts</title><atom:link href="https://fryzekconcepts.com/feed.xml" rel="self" type="application/rss+xml"/><link>https://fryzekconcepts.com</link><description>Lucas is a developer working on cool things</description><lastBuildDate>Mon, 25 Sep 2023 19:42:55 -0000</lastBuildDate><item><title>2022 Graphics Team Contributions at Igalia</title><link>https://fryzekconcepts.com/notes/2022_igalia_graphics_team.html</link><description><p>This year I started a new job working with <a
href="https://www.igalia.com/technology/graphics">Igalia’s Graphics
Team</a>. For those of you who don’t know <a
href="https://www.igalia.com/">Igalia</a> they are a <a
href="https://en.wikipedia.org/wiki/Igalia">“worker-owned, employee-run
cooperative model consultancy focused on open source software”</a>.</p>
<p>As a new member of the team, I thought it would be a great idea to
summarize the incredible amount of work the team completed in 2022. If
you’re interested keep reading!</p>
<h2 id="vulkan-1.2-conformance-on-rpi-4">Vulkan 1.2 Conformance on RPi
4</h2>
<p>One of the big milestones for the team in 2022 was <a
href="https://www.khronos.org/conformance/adopters/conformant-products#submission_694">achieving
Vulkan 1.2 conformance on the Raspberry Pi 4</a>. The folks over at the
Raspberry Pi company wrote a nice <a
href="https://www.raspberrypi.com/news/vulkan-update-version-1-2-conformance-for-raspberry-pi-4/">article</a>
about the achievement. Igalia has been partnering with the Raspberry Pi
company to bring build and improve the graphics driver on all versions
of the Raspberry Pi.</p>
<p>The Vulkan 1.2 spec ratification came with a few <a
href="https://registry.khronos.org/vulkan/specs/1.2-extensions/html/vkspec.html#versions-1.2">extensions</a>
that were promoted to Core. This means a conformant Vulkan 1.2 driver
needs to implement those extensions. Alejandro Piñeiro wrote this
interesting <a
href="https://blogs.igalia.com/apinheiro/2022/05/v3dv-status-update-2022-05-16/">blog
post</a> that talks about some of those extensions.</p>
<p>Vulkan 1.2 also came with a number of optional extensions such as
<code>VK_KHR_pipeline_executable_properties</code>. My colleague Iago
Toral wrote an excellent <a
href="https://blogs.igalia.com/itoral/2022/05/09/vk_khr_pipeline_executables/">blog
post</a> on how we implemented that extension on the Raspberry Pi 4 and
what benefits it provides for debugging.</p>
<h2 id="vulkan-1.3-support-on-turnip">Vulkan 1.3 support on Turnip</h2>
<p>Igalia has been heavily supporting the Open-Source Turnip Vulkan
driver for Qualcomm Adreno GPUs, and in 2022 we helped it achieve Vulkan
1.3 conformance. Danylo Piliaiev on the graphics team here at Igalia,
wrote a great <a
href="https://blogs.igalia.com/dpiliaiev/turnip-vulkan-1-3/">blog
post</a> on this achievement! One of the biggest challenges for the
Turnip driver is that it is a completely reverse-engineered driver that
has been built without access to any hardware documentation or reference
driver code.</p>
<p>With Vulkan 1.3 conformance has also come the ability to run more
commercial games on Adreno GPUs through the use of the DirectX
translation layers. If you would like to see more of this check out this
<a
href="https://blogs.igalia.com/dpiliaiev/turnip-july-2022-update/">post</a>
from Danylo where he talks about getting “The Witcher 3”, “The Talos
Principle”, and “OMD2” running on the A660 GPU. Outside of Vulkan 1.3
support he also talks about some of the extensions that were implemented
to allow “Zink” (the OpenGL over Vulkan driver) to run Turnip, and bring
OpenGL 4.6 support to Adreno GPUs.</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/oVFWy25uiXA"></iframe></div></p>
<h2 id="vulkan-extensions">Vulkan Extensions</h2>
<p>Several developers on the Graphics Team made several key
contributions to Vulkan Extensions and the Vulkan conformance test suite
(CTS). My colleague Ricardo Garcia made an excellent <a
href="https://rg3.name/202212122137.html">blog post</a> about those
contributions. Below I’ve listed what Igalia did for each of the
extensions:</p>
<ul>
<li>VK_EXT_image_2d_view_of_3d
<ul>
<li>We reviewed the spec and are listed as contributors to this
extension</li>
</ul></li>
<li>VK_EXT_shader_module_identifier
<ul>
<li>We reviewed the spec, contributed to it, and created tests for this
extension</li>
</ul></li>
<li>VK_EXT_attachment_feedback_loop_layout
<ul>
<li>We reviewed, created tests and contributed to this extension</li>
</ul></li>
<li>VK_EXT_mesh_shader
<ul>
<li>We contributed to the spec and created tests for this extension</li>
</ul></li>
<li>VK_EXT_mutable_descriptor_type
<ul>
<li>We reviewed the spec and created tests for this extension</li>
</ul></li>
<li>VK_EXT_extended_dynamic_state3
<ul>
<li>We wrote tests and reviewed the spec for this extension</li>
</ul></li>
</ul>
<h2 id="amdgpu-kernel-driver-contributions">AMDGPU kernel driver
contributions</h2>
<p>Our resident “Not an AMD expert” Melissa Wen made several
contributions to the AMDGPU driver. Those contributions include
connecting parts of the <a
href="https://lore.kernel.org/amd-gfx/20220329201835.2393141-1-mwen@igalia.com/">pixel
blending and post blending code in AMD’s <code>DC</code> module to
<code>DRM</code></a> and <a
href="https://lore.kernel.org/amd-gfx/20220804161349.3561177-1-mwen@igalia.com/">fixing
a bug related to how panel orientation is set when a display is
connected</a>. She also had a <a
href="https://indico.freedesktop.org/event/2/contributions/50/">presentation
at XDC 2022</a>, where she talks about techniques you can use to
understand and debug AMDGPU, even when there aren’t hardware docs
available.</p>
<p>André Almeida also completed and submitted work on <a
href="https://lore.kernel.org/dri-devel/20220714191745.45512-1-andrealmeid@igalia.com/">enabled
logging features for the new GFXOFF hardware feature in AMD GPUs</a>. He
also created a userspace application (which you can find <a
href="https://gitlab.freedesktop.org/andrealmeid/gfxoff_tool">here</a>),
that lets you interact with this feature through the
<code>debugfs</code> interface. Additionally, he submitted a <a
href="https://lore.kernel.org/dri-devel/20220929184307.258331-1-contact@emersion.fr/">patch</a>
for async page flips (which he also talked about in his <a
href="https://indico.freedesktop.org/event/2/contributions/61/">XDC 2022
presentation</a>) which is still yet to be merged.</p>
<h2 id="modesetting-without-glamor-on-rpi">Modesetting without Glamor on
RPi</h2>
<p>Christopher Michael joined the Graphics Team in 2022 and along with
Chema Casanova made some key contributions to enabling hardware
acceleration and mode setting on the Raspberry Pi without the use of <a
href="https://www.freedesktop.org/wiki/Software/Glamor/">Glamor</a>
which allows making more video memory available to graphics applications
running on a Raspberry Pi.</p>
<p>The older generation Raspberry Pis (1-3) only have a maximum of 256MB
of memory available for video memory, and using Glamor will consume part
of that video memory. Christopher wrote an excellent <a
href="https://blogs.igalia.com/cmichael/2022/05/30/modesetting-a-glamor-less-rpi-adventure/">blog
post</a> on this work. Both him and Chema also had a joint presentation
at XDC 2022 going into more detail on this work.</p>
<h2 id="linux-format-magazine-column">Linux Format Magazine Column</h2>
<p>Our very own Samuel Iglesias had a column published in Linux Format
Magazine. It’s a short column about reaching Vulkan 1.1 conformance for
v3dv &amp; Turnip Vulkan drivers, and how Open-Source GPU drivers can go
from a “hobby project” to the defacto driver for the platform. Check it
out on page 7 of <a
href="https://linuxformat.com/linux-format-288.html">issue #288</a>!</p>
<h2 id="xdc-2022">XDC 2022</h2>
<p>X.Org Developers Conference is one of the big conferences for us here
at the Graphics Team. Last year at XDC 2022 our Team presented 5 talks
in Minneapolis, Minnesota. XDC 2022 took place towards the end of the
year in October, so it provides some good context on how the team closed
out the year. If you didn’t attend or missed their presentation, here’s
a breakdown:</p>
<h3
id="replacing-the-geometry-pipeline-with-mesh-shaders-ricardo-garcía"><a
href="https://indico.freedesktop.org/event/2/contributions/48/">“Replacing
the geometry pipeline with mesh shaders”</a> (Ricardo García)</h3>
<p>Ricardo presents what exactly mesh shaders are in Vulkan. He made
many contributions to this extension including writing 1000s of CTS
tests for this extension with a <a
href="https://rg3.name/202210222107.html">blog post</a> on his
presentation that should check out!</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/aRNJ4xj_nDs"></iframe></div></p>
<h3 id="status-of-vulkan-on-raspberry-pi-iago-toral"><a
href="https://indico.freedesktop.org/event/2/contributions/68/">“Status
of Vulkan on Raspberry Pi”</a> (Iago Toral)</h3>
<p>Iago goes into detail about the current status of the Raspberry Pi
Vulkan driver. He talks about achieving Vulkan 1.2 conformance, as well
as some of the challenges the team had to solve due to hardware
limitations of the Broadcom GPU.</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/GM9IojyzCVM"></iframe></div></p>
<h3
id="enable-hardware-acceleration-for-gl-applications-without-glamor-on-xorg-modesetting-driver-jose-maría-casanova-christopher-michael"><a
href="https://indico.freedesktop.org/event/2/contributions/60/">“Enable
hardware acceleration for GL applications without Glamor on Xorg
modesetting driver”</a> (Jose María Casanova, Christopher Michael)</h3>
<p>Chema and Christopher talk about the challenges they had to solve to
enable hardware acceleration on the Raspberry Pi without Glamor.</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/Bo_MOM7JTeQ"></iframe></div></p>
<h3 id="im-not-an-amd-expert-but-melissa-wen"><a
href="https://indico.freedesktop.org/event/2/contributions/50/">“I’m not
an AMD expert, but…”</a> (Melissa Wen)</h3>
<p>In this non-technical presentation, Melissa talks about techniques
developers can use to understand and debug drivers without access to
hardware documentation.</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/CMm-yhsMB7U"></iframe></div></p>
<h3 id="async-page-flip-in-atomic-api-andré-almeida"><a
href="https://indico.freedesktop.org/event/2/contributions/61/">“Async
page flip in atomic API”</a> (André Almeida)</h3>
<p>André talks about the work that has been done to enable asynchronous
page flipping in DRM’s atomic API with an introduction to the topic by
explaining about what exactly is asynchronous page flip, and why you
would want it.</p>
<p><div class="youtube-video"><iframe src="https://www.youtube.com/embed/qayPPIfrqtE"></iframe></div></p>
<h2 id="fosdem-2022">FOSDEM 2022</h2>
<p>Another important conference for us is FOSDEM, and last year we
presented 3 of the 5 talks in the graphics dev room. FOSDEM took place
in early February 2022, these talks provide some good context of where
the team started in 2022.</p>
<h3 id="the-status-of-turnip-driver-development-hyunjun-ko"><a
href="https://archive.fosdem.org/2022/schedule/event/turnip/">The status
of Turnip driver development</a> (Hyunjun Ko)</h3>
<p>Hyunjun presented the current state of the Turnip driver, also
talking about the difficulties of developing a driver for a platform
without hardware documentation. He talks about how Turnip developers
reverse engineer the behaviour of the hardware, and then implement that
in an open-source driver. He also made a companion <a
href="https://blogs.igalia.com/zzoon/graphics/mesa/2022/02/21/complement-story/">blog
post</a> to checkout along with his presentation.</p>
<h3
id="v3dv-status-update-for-open-source-vulkan-driver-for-raspberry-pi-4-alejandro-piñeiro"><a
href="https://archive.fosdem.org/2022/schedule/event/v3dv/">v3dv: Status
Update for Open Source Vulkan Driver for Raspberry Pi 4</a> (Alejandro
Piñeiro)</h3>
<p>Igalia has been presenting the status of the v3dv driver since
December 2019 and in this presentation, Alejandro talks about the status
of the v3dv driver in early 2022. He talks about achieving conformance,
the extensions that had to be implemented, and the future plans of the
v3dv driver.</p>
<h3 id="fun-with-border-colors-in-vulkan-ricardo-garcia"><a
href="https://archive.fosdem.org/2022/schedule/event/vulkan_borders/">Fun
with border colors in Vulkan</a> (Ricardo Garcia)</h3>
<p>Ricardo presents about the work he did on the
<code>VK_EXT_border_color_swizzle</code> extension in Vulkan. He talks
about the specific contributions he did and how the extension fits in
with sampling color operations in Vulkan.</p>
<h2 id="gsoc-igalia-ce">GSoC &amp; Igalia CE</h2>
<p>Last year Melissa &amp; André co-mentored contributors working on
introducing KUnit tests to the AMD display driver. This project was
hosted as a <a href="https://summerofcode.withgoogle.com/">“Google
Summer of Code” (GSoC)</a> project from the X.Org Foundation. If you’re
interested in seeing their work Tales da Aparecida, Maíra Canal, Magali
Lemes, and Isabella Basso presented their work at the <a
href="https://lpc.events/event/16/contributions/1310/">Linux Plumbers
Conference 2022</a> and across two talks at XDC 2022. Here you can see
their <a
href="https://indico.freedesktop.org/event/2/contributions/65/">first</a>
presentation and here you can see their <a
href="https://indico.freedesktop.org/event/2/contributions/164/">second</a>
second presentation.</p>
<p>André &amp; Melissa also mentored two <a
href="https://www.igalia.com/coding-experience/">“Igalia Coding
Experience” (CE)</a> projects, one related to IGT GPU test tools on the
VKMS kernel driver, and the other for IGT GPU test tools on the V3D
kernel driver. If you’re interested in reading up on some of that work,
Maíra Canal <a
href="https://mairacanal.github.io/january-update-finishing-my-igalia-ce/">wrote
about her experience</a> being part of the Igalia CE.</p>
<p>Ella Stanforth was also part of the Igalia Coding Experience, being
mentored by Iago &amp; Alejandro. They worked on the
<code>VK_KHR_sampler_ycbcr_conversion</code> extension for the v3dv
driver. Alejandro talks about their work in his <a
href="https://blogs.igalia.com/apinheiro/2023/01/v3dv-status-update-2023-01/">blog
post here</a>.</p>
<h1 id="whats-next">What’s Next?</h1>
<p>The graphics team is looking forward to having a jam-packed 2023 with
just as many if not more contributions to the Open-Source graphics
stack! I’m super excited to be part of the team, and hope to see my name
in our 2023 recap post!</p>
<p>Also, you might have heard that <a
href="https://www.igalia.com/2022/xdc-2023">Igalia will be hosting XDC
2023</a> in the beautiful city of A Coruña! We hope to see you there
where there will be many presentations from all the great people working
on the Open-Source graphics stack, and most importantly where you can <a
href="https://www.youtube.com/watch?v=7hWcu8O9BjM">dream in the
Atlantic!</a></p>
<figure>
<img src="https://www.igalia.com/assets/i/news/XDC-event-banner.jpg"
alt="Photo of A Coruña" />
<figcaption aria-hidden="true">Photo of A Coruña</figcaption>
</figure>
</description><pubDate>Thu, 02 Feb 2023 05:00:00 -0000</pubDate><guid>https://fryzekconcepts.com/notes/2022_igalia_graphics_team.html</guid></item><item><title>Journey Through Freedreno</title><link>https://fryzekconcepts.com/notes/freedreno_journey.html</link><description><figure>
<img src="/assets/freedreno/glinfo_freedreno.png"
alt="Android running Freedreno" />
<figcaption aria-hidden="true">Android running Freedreno</figcaption>
</figure>
<p>As part of my training at Igalia I’ve been attempting to write a new
backend for Freedreno that targets the proprietary “KGSL” kernel mode
driver. For those unaware there are two “main” kernel mode drivers on
Qualcomm SOCs for the GPU, there is the “MSM”, and “KGSL”. “MSM” is DRM
compliant, and Freedreno already able to run on this driver. “KGSL” is
the proprietary KMD that Qualcomm’s proprietary userspace driver
targets. Now why would you want to run freedreno against KGSL, when MSM
exists? Well there are a few ones, first MSM only really works on an
up-streamed kernel, so if you have to run a down-streamed kernel you can
continue using the version of KGSL that the manufacturer shipped with
your device. Second this allows you to run both the proprietary adreno
driver and the open source freedreno driver on the same device just by
swapping libraries, which can be very nice for quickly testing something
against both drivers.</p>
<h2 id="when-drm-isnt-just-drm">When “DRM” isn’t just “DRM”</h2>
<p>When working on a new backend, one of the critical things to do is to
make use of as much “common code” as possible. This has a number of
benefits, least of all reducing the amount of code you have to write. It
also allows reduces the number of bugs that will likely exist as you are
relying on well tested code, and it ensures that the backend is mostly
likely going to continue to work with new driver updates.</p>
<p>When I started the work for a new backend I looked inside mesa’s
<code>src/freedreno/drm</code> folder. This has the current backend code
for Freedreno, and its already modularized to support multiple backends.
It currently has support for the above mentioned MSM kernel mode driver
as well as virtio (a backend that allows Freedreno to be used from
within in a virtualized environment). From the name of this path, you
would think that the code in this module would only work with kernel
mode drivers that implement DRM, but actually there is only a handful of
places in this module where DRM support is assumed. This made it a good
starting point to introduce the KGSL backend and piggy back off the
common code.</p>
<p>For example the <code>drm</code> module has a lot of code to deal
with the management of synchronization primitives, buffer objects, and
command submit lists. All managed at a abstraction above “DRM” and to
re-implement this code would be a bad idea.</p>
<h2 id="how-to-get-android-to-behave">How to get Android to behave</h2>
<p>One of this big struggles with getting the KGSL backend working was
figuring out how I could get Android to load mesa instead of Qualcomm
blob driver that is shipped with the device image. Thankfully a good
chunk of this work has already been figured out when the Turnip
developers (Turnip is the open source Vulkan implementation for Adreno
GPUs) figured out how to get Turnip running on android with KGSL.
Thankfully one of my coworkers <a
href="https://blogs.igalia.com/dpiliaiev/">Danylo</a> is one of those
Turnip developers, and he gave me a lot of guidance on getting Android
setup. One thing to watch out for is the outdated instructions <a
href="https://docs.mesa3d.org/android.html">here</a>. These instructions
<em>almost</em> work, but require some modifications. First if you’re
using a more modern version of the Android NDK, the compiler has been
replaced with LLVM/Clang, so you need to change which compiler is being
used. Second flags like <code>system</code> in the cross compiler script
incorrectly set the system as <code>linux</code> instead of
<code>android</code>. I had success using the below cross compiler
script. Take note that the compiler paths need to be updated to match
where you extracted the android NDK on your system.</p>
<pre class="meson"><code>[binaries]
ar = &#39;/home/lfryzek/Documents/projects/igalia/freedreno/android-ndk-r25b-linux/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-ar&#39;
c = [&#39;ccache&#39;, &#39;/home/lfryzek/Documents/projects/igalia/freedreno/android-ndk-r25b-linux/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang&#39;]
cpp = [&#39;ccache&#39;, &#39;/home/lfryzek/Documents/projects/igalia/freedreno/android-ndk-r25b-linux/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android29-clang++&#39;, &#39;-fno-exceptions&#39;, &#39;-fno-unwind-tables&#39;, &#39;-fno-asynchronous-unwind-tables&#39;, &#39;-static-libstdc++&#39;]
c_ld = &#39;lld&#39;
cpp_ld = &#39;lld&#39;
strip = &#39;/home/lfryzek/Documents/projects/igalia/freedreno/android-ndk-r25b-linux/android-ndk-r25b/toolchains/llvm/prebuilt/linux-x86_64/bin/llvm-strip&#39;
# Android doesn&#39;t come with a pkg-config, but we need one for Meson to be happy not
# finding all the optional deps it looks for. Use system pkg-config pointing at a
# directory we get to populate with any .pc files we want to add for Android
pkgconfig = [&#39;env&#39;, &#39;PKG_CONFIG_LIBDIR=/home/lfryzek/Documents/projects/igalia/freedreno/android-ndk-r25b-linux/android-ndk-r25b/pkgconfig:/home/lfryzek/Documents/projects/igalia/freedreno/install-android/lib/pkgconfig&#39;, &#39;/usr/bin/pkg-config&#39;]
[host_machine]
system = &#39;android&#39;
cpu_family = &#39;arm&#39;
cpu = &#39;armv8&#39;
endian = &#39;little&#39;</code></pre>
<p>Another thing I had to figure out with Android, that was different
with these instructions, was how I would get Android to load mesa
versions of mesa libraries. That’s when my colleague <a
href="https://www.igalia.com/team/mark">Mark</a> pointed out to me that
Android is open source and I could just check the source code myself.
Sure enough you have find the OpenGL driver loader in <a
href="https://android.googlesource.com/platform/frameworks/native/+/master/opengl/libs/EGL/Loader.cpp">Android’s
source code</a>. From this code we can that Android will try to load a
few different files based on some settings, and in my case it would try
to load 3 different shaded libraries in the
<code>/vendor/lib64/egl</code> folder, <code>libEGL_adreno.so</code>
,<code>libGLESv1_CM_adreno.so</code>, and <code>libGLESv2.so</code>. I
could just replace these libraries with the version built from mesa and
voilà, you’re now loading a custom driver! This realization that I could
just “read the code” was very powerful in debugging some more android
specific issues I ran into, like dealing with gralloc.</p>
<p>Something cool that the opensource Freedreno &amp; Turnip driver
developers figured out was getting android to run test OpenGL
applications from the adb shell without building android APKs. If you
check out the <a
href="https://gitlab.freedesktop.org/freedreno/freedreno">freedreno
repo</a>, they have an <code>ndk-build.sh</code> script that can build
tests in the <code>tests-*</code> folder. The nice benefit of this is
that it provides an easy way to run simple test cases without worrying
about the android window system integration. Another nifty feature about
this repo is the <code>libwrap</code> tool that lets trace the commands
being submitted to the GPU.</p>
<h2 id="what-even-is-gralloc">What even is Gralloc?</h2>
<p>Gralloc is the graphics memory allocated in Android, and the OS will
use it to allocate the surface for “windows”. This means that the memory
we want to render the display to is managed by gralloc and not our KGSL
backend. This means we have to get all the information about this
surface from gralloc, and if you look in
<code>src/egl/driver/dri2/platform_android.c</code> you will see
existing code for handing gralloc. You would think “Hey there is no work
for me here then”, but you would be wrong. The handle gralloc provides
is hardware specific, and the code in <code>platform_android.c</code>
assumes a DRM gralloc implementation. Thankfully the turnip developers
had already gone through this struggle and if you look in
<code>src/freedreno/vulkan/tu_android.c</code> you can see they have
implemented a separate path when a Qualcomm msm implementation of
gralloc is detected. I could copy this detection logic and add a
separate path to <code>platform_android.c</code>.</p>
<h2 id="working-with-the-freedreno-community">Working with the Freedreno
community</h2>
<p>When working on any project (open-source or otherwise), it’s nice to
know that you aren’t working alone. Thankfully the
<code>#freedreno</code> channel on <code>irc.oftc.net</code> is very
active and full of helpful people to answer any questions you may have.
While working on the backend, one area I wasn’t really sure how to
address was the synchronization code for buffer objects. The backend
exposed a function called <code>cpu_prep</code>, This function was just
there to call the DRM implementation of <code>cpu_prep</code> on the
buffer object. I wasn’t exactly sure how to implement this functionality
with KGSL since it doesn’t use DRM buffer objects.</p>
<p>I ended up reaching out to the IRC channel and Rob Clark on the
channel explained to me that he was actually working on moving a lot of
the code for <code>cpu_prep</code> into common code so that a non-drm
driver (like the KGSL backend I was working on) would just need to
implement that operation as NOP (no operation).</p>
<h2 id="dealing-with-bugs-reverse-engineering-the-blob">Dealing with
bugs &amp; reverse engineering the blob</h2>
<p>I encountered a few different bugs when implementing the KGSL
backend, but most of them consisted of me calling KGSL wrong, or handing
synchronization incorrectly. Thankfully since Turnip is already running
on KGSL, I could just more carefully compare my code to what Turnip is
doing and figure out my logical mistake.</p>
<p>Some of the bugs I encountered required the backend interface in
Freedreno to be modified to expose per a new per driver implementation
of that backend function, instead of just using a common implementation.
For example the existing function to map a buffer object into userspace
assumed that the same <code>fd</code> for the device could be used for
the buffer object in the <code>mmap</code> call. This worked fine for
any buffer objects we created through KGSL but would not work for buffer
objects created from gralloc (remember the above section on surface
memory for windows comming from gralloc). To resolve this issue I
exposed a new per backend implementation of “map” where I could take a
different path if the buffer object came from gralloc.</p>
<p>While testing the KGSL backend I did encounter a new bug that seems
to effect both my new KGSL backend and the Turnip KGSL backend. The bug
is an <code>iommu fault</code> that occurs when the surface allocated by
gralloc does not have a height that is aligned to 4. The blitting engine
on a6xx GPUs copies in 16x4 chunks, so if the height is not aligned by 4
the GPU will try to write to pixels that exists outside the allocated
memory. This issue only happens with KGSL backends since we import
memory from gralloc, and gralloc allocates exactly enough memory for the
surface, with no alignment on the height. If running on any other
platform, the <code>fdl</code> (Freedreno Layout) code would be called
to compute the minimum required size for a surface which would take into
account the alignment requirement for the height. The blob driver
Qualcomm didn’t seem to have this problem, even though its getting the
exact same buffer from gralloc. So it must be doing something different
to handle the none aligned height.</p>
<p>Because this issue relied on gralloc, the application needed to
running as an Android APK to get a surface from gralloc. The best way to
fix this issue would be to figure out what the blob driver is doing and
try to replicate this behavior in Freedreno (assuming it isn’t doing
something silly like switch to sysmem rendering). Unfortunately it
didn’t look like the libwrap library worked to trace an APK.</p>
<p>The libwrap library relied on a linux feature known as
<code>LD_PRELOAD</code> to load <code>libwrap.so</code> when the
application starts and replace the system functions like
<code>open</code> and <code>ioctl</code> with their own implementation
that traces what is being submitted to the KGSL kernel mode driver.
Thankfully android exposes this <code>LD_PRELOAD</code> mechanism
through its “wrap” interface where you create a propety called
<code>wrap.&lt;app-name&gt;</code> with a value
<code>LD_PRELOAD=&lt;path to libwrap.so&gt;</code>. Android will then
load your library like would be done in a normal linux shell. If you
tried to do this with libwrap though you find very quickly that you
would get corrupted traces. When android launches your APK, it doesn’t
only launch your application, there are different threads for different
android system related functions and some of them can also use OpenGL.
The libwrap library is not designed to handle multiple threads using
KGSL at the same time. After discovering this issue I created a <a
href="https://gitlab.freedesktop.org/freedreno/freedreno/-/merge_requests/22">MR</a>
that would store the tracing file handles as TLS (thread local storage)
preventing the clobbering of the trace file, and also allowing you to
view the traces generated by different threads separately from each
other.</p>
<p>With this is in hand one could begin investing what the blob driver
is doing to handle this unaligned surfaces.</p>
<h2 id="whats-next">What’s next?</h2>
<p>Well the next obvious thing to fix is the aligned height issue which
is still open. I’ve also worked on upstreaming my changes with this <a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21570">WIP
MR</a>.</p>
<figure>
<img src="/assets/freedreno/3d-mark.png"
alt="Freedreno running 3d-mark" />
<figcaption aria-hidden="true">Freedreno running 3d-mark</figcaption>
</figure>
</description><pubDate>Tue, 28 Feb 2023 05:00:00 -0000</pubDate><guid>https://fryzekconcepts.com/notes/freedreno_journey.html</guid></item><item><title>Igalia’s Mesa 23.1 Contributions - Behind the Scenes</title><link>https://fryzekconcepts.com/notes/mesa_23_1_contributions_behind_the_scenes.html</link><description><p>It’s an exciting time for Mesa as its next major release is unveiled
this week. Igalia has played an important role in this milestone, with
Eric Engestrom managing the release and 11 other Igalians contributing
over 110 merge requests. A sample of these contributions are detailed
below.</p>
<h2 id="radv-implement-vk.check_status">radv: Implement
vk.check_status</h2>
<p>As part of an effort to enhance the reliability of GPU resets on
amdgpu, Tony implemented a GPU reset notification feature in the RADV
Vulkan driver. This new function improves the robustness of the RADV
driver. The driver can now check if the GPU has been reset by a
userspace application, allowing the driver to recover their contexts,
exit, or engage in some other appropriate action.</p>
<p>You can read more about Tony’s changes in the link below</p>
<ul>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22253">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22253</a></li>
</ul>
<h2 id="turnip-kgsl-backend-rewrite">turnip: KGSL backend rewrite</h2>
<p>With a goal of improving feature parity of the KGSL kernel mode
driver with its drm counterpart, Mark has been rewriting the backend for
KGSL. These changes leverage the new, common backend Vulkan
infrastructure inside Mesa and fix multiple bugs. In addition, they
introduce support for importing/exporting sync FDs, pre-signalled
fences, and timeline semaphore support.</p>
<p>If you’re interested in taking a deeper dive into Mark’s changes, you
can read the following MR:</p>
<ul>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21651">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21651</a></li>
</ul>
<h2 id="turnip-a7xx-preparation-transition-to-c">turnip: a7xx
preparation, transition to C++</h2>
<p>Danylo has adopted a significant role for two major changes inside
turnip: 1)contributing to the effort to migrate turnip to C++ and
2)supporting the next generation a7xx Adreno GPUs from Qualcomm. A more
detailed overview of Danylo’s changes can be found in the linked MRs
below:</p>
<ul>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21931">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21931</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22148">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22148</a></li>
</ul>
<h2 id="v3dv3dv-various-fixes-cts-conformance">v3d/v3dv various fixes
&amp; CTS conformance</h2>
<p>Igalia maintains the v3d OpenGL driver and v3dv Vulkan drive for
broadcom videocore GPUs which can be found on devices such as the
Raspberry Pi. Iago, Alex and Juan have combined their expertise to
implement multiple fixes for both the v3d gallium driver and the v3dv
vulkan driver on the Raspberry Pi. These changes include CPU performance
optimizations, support for 16-bit floating point vertex attributes, and
raising support in the driver to OpenGL 3.1 level functionality. This
Igalian trio has also been addressing fixes for conformance issues
raised in the Vulkan 1.3.5 conformance test suite (CTS).</p>
<p>You can dive into some of their Raspberry Pi driver changes here:</p>
<ul>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22131">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22131</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21361">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21361</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20787">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20787</a></li>
</ul>
<h2 id="ci-build-system-and-cleanup">ci, build system, and cleanup</h2>
<p>In addition to managing the 23.1 release, Eric has also implemented
many fixes in Mesa’s infrastructure. He has assisted with addressing a
number of CI issues within Mesa on various drivers from v3d to panfrost.
Eric also dedicated part of his time to general clean-up of the Mesa
code (e.g. removing duplicate functions, fixing and improving the
meson-based build system, and removing dead code).</p>
<p>If you’re interested in seeing some of his work, check out some of
the MRs below:</p>
<ul>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22410">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/22410</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21504">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21504</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21558">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/21558</a></li>
<li><a
href="https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20180">https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/20180</a></li>
</ul>
</description><pubDate>Thu, 11 May 2023 04:00:00 -0000</pubDate><guid>https://fryzekconcepts.com/notes/mesa_23_1_contributions_behind_the_scenes.html</guid></item><item><title>Converting from 3D to 2D</title><link>https://fryzekconcepts.com/notes/converting_from_3d_to_2d.html</link><description><p>Recently I’ve been working on a project where I needed to convert an
application written in OpenGL to a software renderer. The matrix
transformation code in OpenGL made use of the GLM library for matrix
math, and I needed to convert the 4x4 matrices to be 3x3 matrices to
work with the software renderer. There was some existing code to do this
that was broken, and looked something like this:</p>
<pre><code>glm::mat3 mat3x3 = glm::mat3(mat4x4);</code></pre>
<p>Don’t worry if you don’t see the problem already, I’m going to
illustrate in more detail with the example of a translation matrix. In
3D a standard translation matrix to translate by a vector
<code>(x, y, z)</code> looks something like this:</p>
<pre><code>[1 0 0 x]
[0 1 0 y]
[0 0 1 z]
[0 0 0 1]</code></pre>
<p>Then when we multiply this matrix by a vector like
<code>(a, b, c, 1)</code> the result is
<code>(a + x, b + y, c + z, 1)</code>. If you don’t understand why the
matrix is 4x4 or why we have that extra 1 at the end don’t worry, I’ll
explain that in more detail later.</p>
<p>Now using the existing conversion code to get a 3x3 matrix will
simply take the first 3 columns and first 3 rows of the matrix and
produce a 3x3 matrix from those. Converting the translation matrix above
using this code produces the following matrix:</p>
<pre><code>[1 0 0]
[0 1 0]
[0 0 1]</code></pre>
<p>See the problem now? The <code>(x, y, z)</code> values disappeared!
In the conversion process we lost these critical values from the
translation matrix, and now if we multiply by this matrix nothing will
happen since we are just left with the identity matrix. So if we can’t
use this simple “cast” function in GLM, what can we use?</p>
<p>Well one thing we can do is preserve the last column and last row of
the matrix. So assume we have a 4x4 matrix like this:</p>
<pre><code>[a b c d]
[e f g h]
[i j k l]
[m n o p]</code></pre>
<p>Then preserving the last row and column we should get a matrix like
this:</p>
<pre><code>[a b d]
[e f h]
[m n p]</code></pre>
<p>And if we use this conversion process for the same translation matrix
we will get:</p>
<pre><code>[1 0 x]
[0 1 y]
[0 0 1]</code></pre>
<p>Now we see that the <code>(x, y)</code> part of the translation is
preserved, and if we try to multiply this matrix by the vector
<code>(a, b, 1)</code> the result will be
<code>(a + x, b + y, 1)</code>. The translation is preserved in the
conversion!</p>
<h2 id="why-do-we-have-to-use-this-conversion">Why do we have to use
this conversion?</h2>
<p>The reason the conversion is more complicated is hidden in how we
defined the translation matrix and vector we wanted to translate. The
vector was actually a 4D vector with the final component set to 1. The
reason we do this is that we actually want to represent an affine space
instead of just a vector space. An affine space being a type of space
where you can have both points and vectors. A point is exactly what you
would expect it to be just a point in space from some origin, and vector
is a direction with magnitude but no origin. This is important because
strictly speaking translation isn’t actually defined for vectors in a
normal vector space. Additionally if you try to construct a matrix to
represent translation for a vector space you’ll find that its impossible
to derive a matrix to do this and that operation is not a linear
function. On the other hand operations like translation are well defined
in an affine space and do what you would expect.</p>
<p>To get around the problem of vector spaces, mathematicians more
clever than I figured out you can implement an affine space in a normal
vector space by increasing the dimension of the vector space by one, and
by adding an extra row and column to the transformation matrices used.
They called this a <strong>homogeneous coordinate system</strong>. This
lets you say that a vector is actually just a point if the 4th component
is 1, but if its 0 its just a vector. Using this abstraction one can
implement all the well defined operations for an affine space (like
translation!).</p>
<p>So using the “homogeneous coordinate system” abstraction, translation
is an operation that defined by taking a point and moving it by a
vector. Lets look at how that works with the translation matrix I used
as an example above. If you multiply that matrix by a 4D vector where
the 4th component is 0, it will just return the same vector. Now if we
multiply by a 4D vector where the 4th component is 1, it will return the
point translated by the vector we used to construct that translation
matrix. This implements the translation operation as its defined in an
affine space!</p>
<p>If you’re interested in understanding more about homogeneous
coordinate spaces, (like how the translation matrix is derived in the
first place) I would encourage you to look at resources like <a
href="https://books.google.ca/books/about/Mathematics_for_Computer_Graphics_Applic.html?id=YmQy799flPkC&amp;redir_esc=y">“Mathematics
for Computer Graphics Applications”</a>. They provide a much more
detailed explanation than I am providing here. (The homogeneous
coordinate system also has some benefits for representing projections
which I won’t get into here, but are explained in that text book.)</p>
<p>Now to finally answer the question about why we needed to preserve
those final columns and vectors. Based on what we now know, we weren’t
actually just converting from a “3D space” to a “2D space” we were
converting from a “3D homogeneous space” to a “2D homogeneous space”.
The process of converting from a higher dimension matrix to a lower
dimensional matrix is lossy and some transformation details are going to
be lost in process (like for example the translation along the z-axis).
There is no way to tell what kind of space a given matrix is supposed to
transform just by looking at the matrix itself. The matrix does not
carry any information about about what space its operating in and any
conversion function would need to know that information to properly
convert that matrix. Therefore we need develop our own conversion
function that preserves the transformations that are important to our
application when moving from a “3D homogeneous space” to a “2D
homogeneous space”.</p>
<p>Hopefully this explanation helps if you are every working on
converting 3D transformation code to 2D.</p>
</description><pubDate>Mon, 25 Sep 2023 04:00:00 -0000</pubDate><guid>https://fryzekconcepts.com/notes/converting_from_3d_to_2d.html</guid></item></channel></rss>
|