Using automation and web scraping on LinkedIn to build your network.
To begin building a bot to interact with LinkedIn, enabling a person to be far more active and contact more people without over analyzing every word, a bot can be super helpful to start some interactions, as not everyone will respond on the site, so why not maximize the chance?
To begin with, we have the following code to automate the login procedure:
This code automates login process on linkedin, user provides email and password as strings, webdriver gets page, as detailed in Web Scraping #1.
To obtain several profiles to scrape, pagination is required, using the following function:
Next, a function can be created to search LinkedIn for people from a certain company or with a specific job title. The follow function uses the preceding pagination function that is used to navigate search results. This returns a dataframe, which will be used later to get in depth information from each profile.
This dataframe contains information that can be used to get a more complete profile on the individual. By creating the df[‘fetch’] column, the complete url is available to obtain each profile as used in the following function.
In addition to the job experience information, a lot of times it’s handy to have the education information, perhaps someone went to school with you or somewhere you’ve previously lived, leaving a great conversation opener. Here is how the school information is extracted:
In addition to the education information, the contact info is extracted using the following function:
To run these and obtain all of the information for multiple profiles returned from the initial scraping function:
These are added to a dataframe, then the first name extracted and used to create a personalized message:
Here, the messages created are iterated through in list format, and each profile receives a personalized connection request.
Data Insights 