--windows functions are really powerful and are somewhat like a group by - except they don't roll everything up into 1 row when grouping. --windows functions allow us to look at a partition or a group ...

Most teams adopt incremental models for one reason. The full-refresh model became too expensive. That is a real problem. A table has 5 billion rows. The daily batch contains 10 million new rows.

This is exactly what LTI Mindtree asked in it’s Senior Data Engineer Interviews recently. With inputs from my students who cracked this offer; here is the list of questions you should not miss. Round ...

--normally when you start the EDA process you have some idea of what you're looking for ...