update dependencies

[antoinerg]

Description

... to pass security scans.

• I upgraded golang to 1.21
• I upgraded go dependencies go get -u && go mod tidy

Upgrade package golang.org/x/net to version 0.17.0 or above.

In particular this one ☝️

See https://github.com/plotly/dekn/issues/6666

EDIT: I can confirm that the CVE-2023-44487 is not detected by Snyk anymore on this image 🎉 We just need to make sure it still works as it did before once integrated in DE5.

[antoinerg]

Testing in https://github.com/plotly/dek-deployment/actions/runs/7702969463

[antoinerg]

cc @BRONSOLO what would be a good way of testing that oomie is doing what it should?

Here are its logs:

│ oomie {"file":"main.go:80","level":"debug","msg":"raw oom event: \u0026{3698196 redpanda 2024-01-29  │
│ 22:07:34.97997535 +0000 UTC m=+16.760453389 /system.slice/docker-a42f881dbba06b6eb0638d062aaec0d71f9 │
│ 4f129c3d1a9b2f407b6304e8ea90d.scope/kubepods/burstable/podba9ab648-bd26-4594-b01e-f843d49884a7/00e78 │
│ aec65b1254858dbc17b6b98e5628d7e57a0cd09af943dfa45e5698f9e71 /system.slice/docker-a42f881dbba06b6eb06 │
│ 38d062aaec0d71f94f129c3d1a9b2f407b6304e8ea90d.scope/kubepods/burstable/podba9ab648-bd26-4594-b01e-f8 │
│ 43d49884a7 CONSTRAINT_MEMCG}","time":"2024-01-29T22:07:35Z"}                                         │
│ oomie {"file":"main.go:92","level":"debug","msg":"not using systemd cgroup path","time":"2024-01-29T │
│ 22:07:35Z"}                                                                                          │
│ oomie {"file":"main.go:80","level":"debug","msg":"raw oom event: \u0026{3049223 redpanda 2024-01-29  │
│ 22:12:26.30406035 +0000 UTC m=+308.084538389 /system.slice/docker-4f85ad99a9877f047da3e1945931515081 │
│ cf09268a6cba0903a3c59f79fcde31.scope/kubepods/burstable/pod577d1b48-4810-4cb6-ab5c-841bbbad978a/b9bd │
│ 109fbb8be62cc0e348a99128a788618336d58131f58af1ddae9bff9247a8 /system.slice/docker-4f85ad99a9877f047d │
│ a3e1945931515081cf09268a6cba0903a3c59f79fcde31.scope/kubepods/burstable/pod577d1b48-4810-4cb6-ab5c-8 │
│ 41bbbad978a/b9bd109fbb8be62cc0e348a99128a788618336d58131f58af1ddae9bff9247a8 CONSTRAINT_MEMCG}","tim │
│ e":"2024-01-29T22:12:27Z"}                                                                           │
│ oomie {"file":"main.go:92","level":"debug","msg":"not using systemd cgroup path","time":"2024-01-29T │
│ 22:12:27Z"}                                                                                          │
│ oomie {"file":"main.go:80","level":"debug","msg":"raw oom event: \u0026{3711324 redpanda 2024-01-29  │
│ 22:27:55.71775935 +0000 UTC m=+1237.498237389 /system.slice/docker-f0cc3d7e411a346e28fbe01b768bac86a │
│ 7c61efd344080c614d9ffc6e7917351.scope/kubepods/burstable/pod214e7c34-2da0-41e3-ac00-66931e5a681a/032 │
│ 872dd112ba59fe36039142292e80be441a97b4447d186284bbc31c706878d /system.slice/docker-f0cc3d7e411a346e2 │
│ 8fbe01b768bac86a7c61efd344080c614d9ffc6e7917351.scope/kubepods/burstable/pod214e7c34-2da0-41e3-ac00- │
│ 66931e5a681a/032872dd112ba59fe36039142292e80be441a97b4447d186284bbc31c706878d CONSTRAINT_MEMCG}","ti │
│ me":"2024-01-29T22:27:56Z"}                                                                          │
│ oomie {"file":"main.go:92","level":"debug","msg":"not using systemd cgroup path","time":"2024-01-29T │
│ 22:27:56Z"}                                                                                          │
│ oomie {"file":"main.go:80","level":"debug","msg":"raw oom event: \u0026{638877 redpanda 2024-01-29 2 │
│ 2:36:20.18199935 +0000 UTC m=+1741.962477389 /system.slice/docker-f0cc3d7e411a346e28fbe01b768bac86a7 │
│ c61efd344080c614d9ffc6e7917351.scope/kubepods/burstable/pod214e7c34-2da0-41e3-ac00-66931e5a681a/99df │
│ c8681691b31ed951f1726a3f9662901b0ad31a99c98b0ab0bcd4dd0e0563 /system.slice/docker-f0cc3d7e411a346e28 │
│ fbe01b768bac86a7c61efd344080c614d9ffc6e7917351.scope/kubepods/burstable/pod214e7c34-2da0-41e3-ac00-6 │
│ 6931e5a681a CONSTRAINT_MEMCG}","time":"2024-01-29T22:36:21Z"}                                        │
│ oomie {"file":"main.go:92","level":"debug","msg":"not using systemd cgroup path","time":"2024-01-29T │
│ 22:36:21Z"}  

Does that seem right?

[antoinerg]

Using a newer version of golang

[antoinerg]

This one in particular was flagged. We needed version v0.17.0 or higher!

[antoinerg]

cc @eff-kay if you have some time could you answer my questions above ☝️

Thank you 🙇

[BRONSOLO]

cc @BRONSOLO what would be a good way of testing that oomie is doing what it should?

Those logs look like oomie is doing the right thing to me (sending the events to redpanda).

As discussed, a full E2E test would involve creating an app that exceeds it's memory limit and seeing that the DE app manager stops the app from running.

[eff-kay]

cc @eff-kay if you have some time could you answer my questions above ☝️

Thank you 🙇

yes the log looks okay. A quick way to test it is to create a workspace or an application. SSH into the pod. Write a recursive function that takes up most of its memory. Once the memory is overflown, an OOM event should be generated in the event log.

[BRONSOLO]

🚀

[eff-kay]

🚀

I would wait until the above test works.

[eff-kay]

cc @eff-kay if you have some time could you answer my questions above ☝️
Thank you 🙇

yes the log looks okay. A quick way to test it is to create a workspace or an application. SSH into the pod. Write a recursive function that takes up most of its memory. Once the memory is overflown, an OOM event should be generated in the event log.

Once the event is generated. Then its the backends jobs to parse that event and show the relevant warning to the frontend.

[eff-kay]

@antoinerg use the following demo attached.

1. Create an app.
2. Then use the code from the attached demo, and push it. It should consumed 7.5 GB of RAM. The default limit for all apps is 8GB.
3. Then go inside the pod of the app. Open the app.py file in vim and modify mem_limit = 7500 to 8500 or something higher than 8000.
4. Restart the gunicon worker by executing pkill -HUP process in the terminal. It should restart the app with the new limit. This limit crosses the 8GB boundery and should evict the pod.
5. Verify that an OOM event was issued on the kubernetes event log.

app.zip

[antoinerg]

Thanks @eff-kay for the reply!

I got the message in the frontend: