I have two sets of data. and I want to join them by two key values (storm_ID, Cell_ID).
The first dataset looks like:
Storm_ID,Cell_ID,Wind_speed 2,10236258,27 2,10236300,58 2,10236301,25 3,10240400,51
The second dataset looks like:
Storm_ID,Cell_ID,Storm_surge 2,10236299,0.27 2,10236300,0.27 2,10236301,0.35 2,10240400,0.35 2,10240401,0.81 4,10240402,0.11
Now I want an output which looks something like this:
Storm_ID,Cell_ID,Wind_speed,Storm_surge 2,10236258,27,0 2,10236299,0,0.27 2,10236300,58,0.27 2,10236301,25,0.35 2,10240400,0,0.35 2,10240401,0,0.81 3,10240400,51,0 4,10240402,0,0.11
I tried join command in Linux to perform this task and failed badly. Join command skipped the rows which didn't match in the database. I can use Matlab but the size of the data is more than 100 GB which is making it very difficult for this task. Can someone please guide me on this one please. Can I use SQL or python to complete this task.