Thursday 10 December 2020

Release 0.4.3 - The End of Something Great


Hello my wonderful followers!

It is here... It is finally here. The end of this nightmarish semester is finally here.
I have to say the past two weeks have been really tough for me, with all the final assignments/ labs/ projects.... AND exams bearing and breathing down my neck as the due dates approached.

Needless to say, I got through it. Pulled a couple all nighters and multiple 14/16 hour days worth of work. It feels GOOD though. I found this semester to be the hardest/ most amount of work that needed to be done. Yet, I finished and cannot express how many things I have learned over the course of this semester. 
I have to give my koodos to the very knowledgeable and helpful professor HumphD (OSD600 - Open Source professor). Without him, and this course, I would never have entered the world of open source and have my transition into becoming an open source dev/ contributor go so smoothly. 

Anyways, back to the point. For my last PR of this wonderful course, I chose to do yet another issue for the repo of automating whatsapp web platform. This one was tougher than the last or any previous issues I've done for this repo. 

I think my biggest downfall is finding out how to get python and selenium to collaborate with me without an issue.

So. Let's start by explanation of what I had to do.

I had to open whatsapp. Once connection was established through the bot, I had to cycle through all individual chats and open them.
Once opened the chat info drop down had to be clicked and navigation to delete chat (of the opened already chat) was to be made. Once over the delete element (which is found using a CSS Selector), the element had to be clicked. 

Clicking the "outter" delete would trigger another pop-up box/event to be opened. The bot would have to wait for the presence of the element to be available. Navigates to the element using a CSS Selector and clicks the element.

I ran into problems with the above step because the bot would hang even after the element appeared. I contacted my friend Navpreet (maintainer of the repo) and asked him why this was happening. He quickly glanced at my code when we were on discord with him (late at night because he lives in India) and he explained I was not waiting for the element to become available. So as I mentioned above. The bot would have to wait for the presence of said element "delete" and then execute the follow up functions. He told me it was hanging because it was throwing an uncaught exception and did not know what to do after the exception was thrown. Interesting stuff to be honest. I would have to remember that for next time I contribute to his repo, or other projects :D

After adding that little function in. All I had to do was close the chat info with the function "close_info()". I tested it one final time and made my PR.

I noticed a small problem with the bot behaving strangely on other functionalities that were implemented previously.

I noticed that some of them were not working how they should have been and mentioned it to Navpreet. He said he is aware of this issue and it was happening because of whatsapp changed their CSS Selectors. Thus making pretty much all the main functions break since the selectors did not exist anymore. He thanked me for noticing it as well and said he will make an issue about this when he has time, or fix it himself. 

To be honest, I will probably help him over the winter break since we are buddies now.

Well. Now that I am done with OSD and all my exams. Which by the way could not have ended on the most perfect date than the 9th of December... because CYBERPUNK 2077 came out. AND let me tell you guys. WOW. what an amazing game it has been so far. I have only played about an hour because I was dead tired yesterday, but I was at a loss for words. 8 years in the making was well worth the wait.


I would like to thank professor Humphrey one last time this semester for giving my colleagues and I the tools and knowledge to gain tremendous amounts of experience this semester and work on whatever we wanted and found enjoyable. 


For the last time this semester...

Till next time, 
Gossip Plamen

Thursday 3 December 2020

Release 0.4.2 - Progression Tree


Hello fellow programmers/ followers.
I sincerely hope this past week has been treating you great, because I sure have gotten stomped to the ground due to various things in my academic career -cough- Game Dev... #ThanksGeorge
I also got shot down hard by someone in my personal life. But that is too personal for the internet.

So anyyyyways.......

This week is a little bit of a continuation of the plans I made for my open source development ventures last week. A little bit of a recap.. Last week I decided to pick a repo that I have contributed to in the past. Seeing as my schedule is super tight until the end of the semester, I decided to go back a month and revisit the automating whatsapp web platform. I picked two issues because one was easier than the other and I had created something similar.


So I saw that most people on the issues that have been assigned have made little or no progress. I asked Navpreet (the owner/maintainer of the repo) if I could take this issue. He gave me the green light. Although, to be honest I already had started working on the issue before he said it is fine (I knew it would be fine). 

I was tasked with creating a function similar to one that already existed. I had to create a function to remove Administrative privileges from a person/people in a whatsapp group.

As I began working I first examined the steps I had to manually take to remove admin privileges in order to replicate them and adding them into the automation bot.
Since the main logic is being done by Selector constants I had to find the selectors that were responsible for removing admin status from a person in a group.

I inspected the element with the chrome dev tools and quickly found exactly what I was looking for (this took me a while in Release 0.2, but I knew what to look for now).

P.S. The steps you take to removing an admin all lead to using the same selector regardless.

Now that I had my selector. It was time to actually put it in the constants .py file that is used for everything.
GROUPS__REMOVE_ADMIN = (By.CSS_SELECTOR, '._1OwwW ._3oTCZ[title="Dismiss as admin"]')

After that was done it was time to start writing the actual code.

The first thing I had to do was to iterate through the groups and find group name with the passed in one. This is being done by querying the selector and looking for it to match with whatever element you are on. Once the group chat was open, the bot needed to open the group chat info and navigate to the participants section. From there it searches for the name(s) passed in to remove admin privileges from, this is done by finding if the admin logo is present on their name. Clicks their name box and selects "Dismiss as admin". 

Everything worked. there was a couple of bugs at first. I was not checking if the user of the group was already an admin or not and the bot started hanging because it was trying to find an element to click on that did not exist yet. Another problem I actually ran into was that I had to tweak the CSS selector a bit because it is the same selector for making an admin and removing an admin. This is why in the above snippet I decided to include the title as well so it does not create an admin by accident.

Now, that I was done with this feature I could hurry back to Navpreet and have him run it on his end. I submitted a PR and since Navpreet and I are somewhat acquainted from my previous contributions, he knows my secret love most hated feeling is towards python and everything python related. So he gave me a little praise to help me with the next big issue I was going to do for him.

Everything is going swell. My PR even got merged. ONTO better things... not really. OSD is probably my most enjoyed course right now. Everything else feels like such a drag and there is sooo much work.

Anyways this is where I end things for my ventures of open source this week. I have actually started working on my other issue for Navpreet but it is proving to be a little more difficult than I expected. 0.4 amiright?!

I am now going to crack open a beer and enjoy the rest of my night, and try not to think about stupid little details.

Till next time,
Gossip Plamen

Friday 27 November 2020

Release 0.4.1 - Planning my life (for the next 2 weeks)

 What's going on good people?!?!?!

These next couple of weeks are super busy for me so I will be making this one short, unlike the last blog post I made.

So this past week in our open source class we were tasked with picking an issue(s) that would need to be solved and PR(s) submitted for 0.4 at the end of the semester.

This post will be to describe my planning process/ thoughts on what I would like to do over the next couple of weeks. I thought I would try and keep it simple and go back to a project that I have worked and made PRs for in past releases 0.2/3. 

I will be going back to automating the web social media platform of Whatsapp. This will be the repo I will be picking my issues (i.e. issue# 1, 2,**Might change depending if I get them assigned, or if I find more interesting ones**) from. I see a couple of issues that have been stale (no progress being made) for a while now. I would like to work on them so that the maintainer/contributors could add more features and move on. Pretty much all the issues are linked together in terms of usability with one another.

Well, That is pretty much it for this one. I will be starting to work on the issues once I finish my course work for GAM531/537 (sigh... #ThanksGeorge)

Till next time, 
Gossip Plamen

Tuesday 24 November 2020

Testing to Pass the Time

What is going my good people of open source dev?

Have you ever dreaded those damn tests in school? Have you always put your heart and soul on the line to get that 'Pass'? Yeah, me neither...

This week's lab in my open source class we were tasked with creating unit tests for.... 
**queue drum-roll**
Our link checker program... Yes that again. Those of you who have no idea what is going on, good... you don't need to know. DeadLinkage (just incase you are interested)

As I mentioned, we had to integrate unit testing in our link checking programs. Let me start off by saying #$%! unit testing on java... OH MY GOD! What a pain in the ass this lab was.
I mean I loved the way java handles everything... But for real though. Unit testing on my code was a nightmare. I spent roughly 3-4 days trying to get JUnit (the testing framework for java) to cooperate with my IDE. 
It was a mistake primarily because everything had to work on a CLI. 
Anyways, at first I tried to get stuff working with JUnit5 everything was fine and dandy up to a point where I had to mock objects and classes such as the HttpUrlConnection and URL.

This posed a problem because the mocking API that I had settled on using was mockito. There was not a whole lot of documentation and examples of Mockito cooperating with JUnit5 so I decided to backtrack a version of JUnit to 4.12.

A good chunk of my time was spent walking back and forth while reading documentation on my phone about how mockito works. On my off time I would try code only to have it fail miserably.

I decided to take my problems to stack overflow and see if anyone else has run into this issue. I spent about a day reading through stackoverflow pages/questions and came to the conclusion that mockito, while a powerful mocking tool. Could not mock Final classes, the exact classes I was trying to mock (of course...)

While reading similar problems from other people trying to use mockito to mock final classes I came up on a forum that described an API that was used in parallel with mockito, Powermock. Powermock had an API of its own that works with the mockito API and served as sort of an extension to it.

So began my readings of the novel called "The Documentation". The way powermock is used was very similar to the way mockito is used to mock. Actually they were almost identical, slight variations from version to version in syntax, but that didn't pose much of a "threat". 


Theoretically, it should work right? WROOONG!
I was so close to making it work until I ran into another problem. Version compatibility.....

The first thing I did was ask this cool girl (who unrealistically also likes Java like me)... and a lot of other things but anyways. Nesa had no idea what the issue was and proved to be of no help. She actually said she hadn't started the lab herself so she had no idea what to do. Here is a little proof.

Thanks for the compliments Nesa, but nice habits don't write the unit tests. (or at least not yet)

So, powermock 2.x is not really compatible with most versions of mockito. The same goes for the opposite. A full compatibility list can be found here. (note: I did not find the compatibility list right away) Learn from my mistakes and get the right versions.

Awesome! I found the right versions, downloaded the correct JAR files to stash in my external library list and now it was time to write some tests. 

For my first unit test I picked an area of my program that would be easy to work with and wouldn't be dependent on many mocked objects that needed to be passed in. That area was my function to load an ignore-urls.txt file. The only parameter that this function needs is the filename which is given through the system arguments (which are stored as Strings) when the program is ran. I had to mock the file name being passed in which was as simple as making a string hold a "filename.txt" and running the function with that argument. The function returns an array list of strings which I then had to iterate through a loop and assert that they were returning the correct thing. Easy peazy.

Hazaah! and so I had written my first unit test. I ran the test (remember in my IDE) and everything was working correctly. I decided to try and run the test on the CLI. aaaaand more problems started occurring. I was not able to compile my test java file due to the terminal that I was using not being able to see environment variables. I took my problems to the very helpful and knowledgeable professor HumphD. 

What do you know, the professor was right. Everything compiled fine in command prompt and another version of PowerShell. Thank you for the help! I was pulling my hair out and almost ended up looking like my buddy Chris.

It compiled, sweet! I only hoped that was my biggest issue at the time. I tried to run my compiled Test Runner only to have it crash and burn for missing dependencies. There was way too many for me to add and I did what every sensible person would do. 
I explained what the problem was and David (yeah first name basis, we are cool like that) told me something I dreaded of hearing ever since I started this damn link checking program at the beginning of the semester. 

Make a build project!" That is all I was hearing and my ears were ringing like I had been standing next to a speaker at a rave/club (ah what simpler times we lived in before all this pandemic stuff started happening).

Well I guess it was time to do the right thing. It was time to use a build tool like maven to "manage all this crap" as HumphD put it. 

Integration into a maven project really was not that hard. The IDE (IntelliJ) I use actually makes it really easy for you to setup a maven project structure on a Java project. I will link it here, in case others are trying to do the same.

Once maven support was added into my project I began to add the dependencies from the maven general repository, where you can find almost anything and everything. It is extremely easy to add a dependency into a Java project with maven... Literal click of a button.
Anyways once everything was setup, I ran the test I had written from before (The maven way) on cli and everything compiled and ran PERFECT! I was so happy I got over this hurdle and could finally move on after 3 days of pain and torture on my way back from the depths of hell.

Oh man, maven opened my eyes to a whole new world with Java. Maven is now my new best friend. Everything is so easy with this build tool.

Anyways, I moved on and started to write my tests for the core of my program. Testing my CheckLink logic. These functions basically take in a URL (string) to be checked and a HttpUrlConnection to get the status code of said URL. Remember 10 minutes ago when I was explaining about mockito and powermock being used together to mock final classes such as HttpUrlConnection? Well, this is what I needed this mocked connection for. I had to mock the status codes for a specific url, I chose because why not. I set the status code of the mocked huc to 200 and 404 so that they would pass my goodLink and badLink functions and basically checked the output the function returned against an expected output I had preset. This was not that bad once I already had the knowledge of how to do this. Once I finished writing the tests I ran all tests and to my amazement everything works flawlessly.

I am so glad I spent a couple of hours rebuilding my project as a maven build. PHEW. In the long run it probably saved me so much time.

Next on the task list was to incorporate a code coverage tool. I chose to work with JaCoCo because it was the first one that popped up on google it was in the maven repository and was easy to work with. Guess how long it took me to get it up and running. Roughly 15 seconds!

MAVEN BAYBEEHH! but for real I just copy and pasted the JaCoCo dependency code into my pom file and specified the directory I wanted my new reports to go in and ran my tests. Boom I now had code coverage (although not a lot of code was covered with 3 tests LOL)!

Next up on the task list was to add integration testing via github actions. This was not hard at all. Github actually has already made templates for executing maven tests. I just had to change the platform that the project was being built/ran on from ubuntu to windows although I doubt that makes a difference anymore since I added maven support. 
If you are interested in my workflow file it can be found here.

All I do in this workflow is the following:

  • Setup Java
  • Cache dependencies
  • Compile the project
  • Run tests
Running tests is dependent on whether the project compiles or not. If the project does not compile there is no point in running the tests. That is it. Pretty simple. Github documentation on maven CI was probably the clearest documentation I read throughout this whole lab. 

Anyways this is how executing the tests look like on each commit to master.


The next item on the agenda was to write some tests for a fellow class mate. I did not even give it a thought I messaged my bald headed friend Chris right away to ask if he wanted to work together on this lab as well. To my surprise he said sure. I really did think he hated me.

Anyways, as I have expressed my feelings about Python before (utter hatred). I asked Chris to give me a little crash course on using pytest to see how everything works. He did a great job of explaining everything. We then discussed what exactly he wanted me to write tests for. He mentioned that he would like tests for the functions that print out each status code...(like why was this necessary for you to make Chris) uhm yeah sure let me do that.

So I decided to add the following tests to ensure they were returning proper status codes and that nothing was broken. If these functions stopped reporting the correct code the entire program will break. so anyways this is what I came up with.

def test_status_check_unknown(capsys):
  • this function is used when a link is unknown
  • I had to figure out how to capture stdout and read what was being returned from a test
  • I also had to find the ansi codes to verify the texts match from stdout since the text color was white
The next test I just made a mockresponse() class and assigned a status_code of 300 and checked that the object had a status_code member which was set appropriately. I did this because the 'requests' library in python returns an object containing these details so I mocked that. 

The rest of the tests are more or less the same, I just had to ensure the ANSI code was correct. 

I tested this on my end using the Pycharm IDE, I also changed the function I was testing in question and the tests did in fact fail because the underlying functionality of the base function failed. 
This reminds me of the matrix checker for our submissions in IPC/OOP/345.


Overall this entire process was a nightmare, I would never want to repeat it again. 

HOWEVER, with that being said I learned so much about java, build tools, unit testing and integration testing. It was a really big learning curve to say the least but it is now done and I can finally do some Game Dev work for my final project. Thanks for the learning experience David. It really was a lot of involvement and reading on this one cheers.

Working with Chris was always a sad reality that I had to settle to work with this bald little man 😐 LOL I'm joking Chris, it was a pleasure as always.

Till next time, 
Gossip Plamen

Saturday 14 November 2020

Release 0.3.2 - Don't Sit Too Close to the VR TV


Do you remember when you were young and your mom yelled at you for sitting too close to the TV?!

Well, I do too. It was annoying to say the least. I can't blame her though it was a bad habit that was part of most of my childhood, especially when playing my PS1 and PS2, the good ol' days. 

So why doesn't she yell at me whenever I use my Oculus Quest VR? It probably has something to do with the fact that I am 25 years old now and it looks sort of silly for a grown man to get yelled at by his mommy... 


So, as mentioned in my last blog Release 0.3 has two parts, or rather 2 PRs to be submitted.
I wrote about how I contributed to Telescope in my last blog. That was fun.

However, I needed a new repository to contribute to this week. Since I am a big VR enthusiast as I have said multiple times in previous blogs. I chose to do something related to VR.

I started searching and came out empty on specific things that I wanted to do. I then realized that one of the most used programs that I personally use almost on a daily basis is the ALVR application.

ALVR is an open source remote VR display for Oculus Quest. With it, you can play SteamVR games in your standalone headset.

I went on the github repo and found an interesting issue that I felt would benefit not only me, but a lot of people.
I found an issue to translate the WHOLE application, since my native tongue is Bulgarian I decided to submit a PR for the Bulgarian language.

Eh, a translation PR who cares, that is easy. 
You would think so but... well it was relatively easy but it was a lot of work, especially since my Bulgarian has been degrading over the past 18 years since I came to Canada.

I began to look through the files on what needed to be translated.
Thankfully for me, the structure of the project was really well organized and finding the language support files was not that hard.

The first thing I did was to copy the "default" English files into a new folder with the language code "bg" to use them as the base for my translation files.
I then linked those files with the default ones to change when the user selects a language from the drop down menu.

All that was left was to translate. OH SO MUCH TRANSLATION!

I literally spent over 7 hours alone just for translation of values and text. It was tedious work but my PR got merged after about a day ,and a review from the maintainer, later. I was excited because I had contributed to something used all around the world.

After my PR got merged I immediately sent the latest version to two of my friends (who are also Bulgarian) for them to test the translation. They commended me on a job well done.

 I showed all my relatives because this is one contribution that I was extremely passionate and proud of.

Here is a meme, just for fun (because I had it laying around and wanted to post it).

Welp, that was all from me for 0.3. whoever has a VR headset go checkout ALVR and star it. It needs more recognition.

Now, I am going to bury my head in Amazon and Canada Computers because I am building myself a brand new PC for gaming... writing code O.o

Till next time, 
Gossip Plamen

Release 0.3.1 - The Seneca Hubble Telescope


Hello once again today.

I will just get to the point of this blog.
So Professor Humphrey gave us release 0.3, 2 weeks ago (I think), I've sort of lost track of time. This 0.3 release was to contribute to two more repos.

The first being a contribution to the Telescope repo.
Telescope is an open source web server and client application for aggregating and presenting a timeline of Seneca's open source blogs. Telescope makes it easy to see what's happening with open source at Seneca right now.

So the first thing I had to do was to find a Telescope issue that I can actually do...
My JavaScript really sucks major donkey balls. So I looked and looked through the hundreds of issues (at the time) that had been filed and stopped at the perfect one for a front end bug that had re-occurred due to a commit changing the flow of how posts are being handled.

I got down to business and started hunting down the culprit CSS that was responsible for not making the posts on mobile not extend the whole screen width. In all hind sight I probably spent a lot more time than required for something like this because as mentioned above, my JS is horrible. It also doesn't help that I am not a big fan/ have much interest of web technologies.

The problem that was at hand was that because of commit #1174 and soon after #1217 the content was not being displayed properly as maintainers/ alumni want.
As I said I hunted down the culprit and found that the default CSS style was not being overridden. I simply applied the style so that it will be used and everything worked according to plan.

Although, this was not a huge fix. I learned a lot about collaboration on a big project and the multiple checks that have to be conducted before something is merged and eventually deployed. 

My PR actually fixed two issues at the same time, the second being by accident but that is not important O.o

These were the results of my small bug fix.
                      Before:                                                  After:

Oh and the search page issue (linked above)

I was pretty happy with the changes and how it turned out.

Well that is everything for now.

Till next time, 
Gossip Plamen

Formatting and Linting my Life


Hello my fellow followers (professor).
I hope you are all doing well, despite Toronto being in the red zone. 

So, this week was an interesting one for my open source ventures. Lab 7 was released and we were tasked with implementing a code formatter and linter for our, you guessed it, link checker.

To be honest, I've grown fond of working on this little program because I enjoy building on top of it. It's awesome, it started as a POS horrid bandaged looking program but has since turned into something gorgeous, something like a rose if you will.

Anyways, I will get right into it because I have to finish 2 more blogs for this week and do my Game Dev lab #ThankGeorge. 

The first task was to choose a code formatter that would format code according to some standards. I chose the google-java-format that professor Humphrey suggested.

So this tool was actually pretty easy to use. I read the manual and just ran the jar file with the options on the CLI and it worked like a charm.

Low key,
I kind of don't like the way it formatted some of the files and in some instances it became even harder to read... Google standards are weird.

I actually took the extra challenge and made a git hook that will do this automatically every time on any file you commit. (I had some help from a couple of google searches on how to set this up properly)

So I made a little script that would download the formatter to a temp directory, if you don't already have it. The rest is self explanatory but I will throw the script anyways.

I actually installed this as a plugin for my IDE because... why not.

So the next thing we were tasked with was to pick a linter for our code that would improve our code and spot bugs automatically, professors lectures were actually really neat on this. I won't get into details though.

Once again, I chose to go with the professor's recommendations and chose the SpotBugs linter for java.
This was a big harder to use as the documentation was not that explanatory. As the standalone version I chose to use the GUI to see how it was. It was pretty cool so I recommend using that. It spot a couple of "bugs" in my code but not that many. I would say the major one it found was in my Load Ignore files, java file.
I was not closing the BufferedReader and it told me that (see below)

This is actually the same linter but the plugin version on my IntelliJ IDE. The fix it suggested was to either close the reader manually or encapsulate it in a try and catch block, like so.

You can read the full documentation on the on my repo.

The final step was to squash all of my commits and push it to github.

Alright, that is all from me for this week's lab.

Till next time, 
Gossip Plamen

Release 0.4.3 - The End of Something Great

  Hello my wonderful followers! It is here... It is finally here. The end of this nightmarish semester is finally here. I have to say the pa...