i trying calculate datediff , count_diff in pyspark on event data.
the data looks
deviceid techid name count load_date m1 1 30 23-01-2016 m2 1 b 40 23-01-2016 m1 1 45 29-01-2016 m1 2 50 30-01-2016 and want
deviceid techid name count load_date datediff countdiff m1 1 30 23-01-2016 na na m2 1 b 40 23-01-2016 na na m1 1 45 29-01-2016 6 15 m1 2 50 30-01-2016 na na how create column these values in pyspark, taking datediff on change of event condition.
