THE SCRIPT

    I finally managed to create a script that works. It is an implementation of CQL in Python. I spent so many days working on this script, and I encountered errors after errors. I have a total of 17 thousand books that I have to find the country and region of publication for. I am using the library of congress to collect the data with the Search/Retrieve via URL, a Contextual Query Language. With such large data, it is hard to debug as I never know when an issue may occur. Also, CQL is a language that does not really have a big online community, so finding solutions online was hard. Hence, I had to spend at least 2 days fixing a single bug. The biggest issue I had was a title not being supported by CQL even if the title was in string format. After spending hours figuring out why a string is not considered a string, I found out that there is a difference between string and string literal; I do not even understand the difference, but at least I found out that python has a feature called "repr()" that converts a string into a string literal, so it solved my issue.

    So far, working on this particular script made me realize how work is done in the real world. Most of the time, when I do a personal project, I create something based on the knowledge I already have. My end goal is determined by the skills I already have. But with this project, I have an end goal and I have to reach it at all cost. It means that I will have to go through some things that I have never seen before, or things that I do not know. On top of that, I have a deadline, so I have to do the work and learn anything I need to learn in a limited time. This experience helped me strengthen my mindset and allowed me to handle problems calmly and in a professional way no matter how stressful the situation may be.

Comments

Post a Comment

Popular posts from this blog

FINAL TASK

       Today, I started what probably is my last work with Project Gutenberg. I am working on a named entity recognition. This is my second time applying timed entity recognition to this data. During my previous task, I used SpaCy to do the NER, but this time I am using JohnSnowLabs. JohnSnowLabs is a website with a collection of pre-trained machine learning pipelines. This time I am using the BERT pipeline to accomplish my task. The final result of my work will be a topic modeling and a named entity recognition which my supervisor will then analyze.      I think it is the right time to reflect on some of the things I accomplished during these nine weeks of work. I have learned a lot from this experience, whether it is a way to complete work, a learning habit, or an ability to multitask, I have gained new skills, knowledge, and work experience. I believed that these gains will allow me to perform well in my future internship.
Read more

FINAL POINT

       I recently completed my internship at Vanderbilt University, where I worked with an English professor and a librarian for a text-mining project. The goal of the project is to help the English professor in his research about how Law is represented in general dialogue during the Romanticism period (a period of literature between 1750 and 1850). So, the data collected were English books published in the United Kingdom and Ireland between 1770 and 1835. The main source of data was Project Gutenberg, an online library of free eBooks.       We faced a load of challenges while working on this project. One of the biggest issues was figuring out the correct book to collect. While project Gutenberg provides some information such as authors and editions of the book, it does not provide the year of publication, so there was no way for us to know what the right book is. We have attempted to solve the problem by collecting all the books in English and fr...
Read more