Saturday 31 October 2020

Release 0.2.4 - More Pythonic Automation

 


Hello again, my fellow programmers.

The end of the month of October is here and a fourth and final PR has to be made in order to fulfil the requirements of my 0.2 Release.

I will keep this post short and sweet (again)... because it is Halloween and I have candy to eat... erhm give out O.o 

So, this week as I mentioned above I had to complete my final PR for Hacktoberfest. I was stumped on what I actually wanted to do and contribute to as I was also pressed for time due to spending time on my ever growing open world concept level for my Game Dev. course. #ThanksGeorge

I searched and searched for something interesting to do with knowledge I already had. I then remembered that the point of each consecutive contribution was to learn something new. I remembered the Selenium/Python automation bot from my previous PR and saw that there was open issues the owner of the repo wanted help with. I picked one where I needed to scrape a phone number from a specific "open" chatroom in whatsapp web.

I got to work and I completely forgot about asking the maintainer to assign me to this problem, which I think is a big NO NO in Open Source Dev. 

So, I thought that since I used selenium in my 0.1 release to scrape a web page for links using Java, it would be similar. 

I mean it was not that different but the way the maintainer does the scraping is weird and I was not familiar with it. He finds/scrapes data by using the CSS selector class. Which I found to be an interesting approach, it is risky because whatsapp can change the layout and selector class and that can cause everything in the bot to break. But, I am not one to judge.

Nevertheless, I started searching how to scrape exactly what I was looking for:


Now, that I had the exact selector for the specific class I put the parent div selector class into the constant file class that Navpreet (owner of the Repo) had previously made.

In order to scrape the telephone of a specific chatroom with a person I had to:
  1. Use his already made function to Open a chat with a specific person
  2. Open the chat info
  3. Scrape the phone number from the person's info
  4. Close the chat room
  5. Return the scraped phone number
I ended up using a lot of already made functions throughout his code to avoid code duplications where it was unnecessary, and finished with something looking very simple and clean.

I tested and everything ran fine on my computer... KEYWORDS: "My Computer"
I made a PR thinking everything would be fine.

Navpreet reviewed the code and merged it. YAY!

He then messaged me on discord a couple of hours later saying that there was a bug scraping the number. This was happening because the element that contained the phone number was not yet loaded but the bot scraped something that did not "exist" yet... I have no idea what could be causing this since everything ran fine on my end.

Eh, whatever, I hopped on a call and we started debugging and we thought to implement a "wait" functionality for the bot to hang and wait for the element to be available in order to be scraped.

It worked!
Navpreet made a PR addition to my PR and everything was fine and dandy. 

He is actually a really cool dude. He lives in India and his English was perfect! I was surprised. We exchanged a couple of stories and laughs after we finished debugging. Needless to say I made a friend all the way across the world. Awesome feeling when you can connect with someone on a personal and intellectual level.

Anyways I am off to go HaNd OuT candy and watch a bunch of scary movies. 

Till next time, 
XOXO, 
Gossip Plamen

No comments:

Post a Comment

Release 0.4.3 - The End of Something Great

  Hello my wonderful followers! It is here... It is finally here. The end of this nightmarish semester is finally here. I have to say the pa...