What are the pain points your are facing with data cleaning? How do you handle it for now?

Data cleaning depends on the problem domain.

Compare output from a spoctrometer (or spectrograph) vs. eliminating outliers from an almost linear process. One will wreck your data and the other is the only correct thing to do.

         *         

**** ****

Why? What's currently annoying about notebooks that you have to deal with compared to just directly going to users?

Ah, well, rereading your original post I realize now this isn't necessarily painful for me. Perhaps though, the annoying aspect is seeing others use proprietary excel spreadsheets without a data lake. Conway's Law?

Does VS here mean Visual Studio? I would not call myself a data engineer, I just play one at work sometimes. Many hats, yknow?

"the annoying aspect is seeing others use proprietary excel spreadsheets without a data lake" => what's painful about that?

VS = compared to, versus

Hah okay. I read VS different from vs. The pain, in part, is hidden functions, rarely ever inline documentation, difficult to reuse or repurpose, Windows-centric, etc.

Thanks for your feedback, I can understand that it's not enoyable to record a video.

But for now we will keep these steps and this process since it's important for our recruitment process.

Thanks for the feedback.

I can understand that it can feel like an investment to apply.

But for now those are important steps we need in our recruiting process.

Let me know if you have any other feedback :)

Thanks for your brutally honest feedback. What makes you think it's shit-tier company? How do you evaluate that?

Thanks for the feedback. Indeed I don't want to get the wrong expectations & be as clear as possible. It will be a painful cleaning job at first (at least that's how refactoring is viewed by most).

And yes we do have a carefully crafted process. Those questions / requests are very meaningful for us.

Recording a video of yourself explaining something is just a non-starter for me. Maybe I'm just old and not part of the Instagram/TikTok generation, but I'd feel very self-conscious. I doubt I'm the only one.