CodeIgniter’s Query Builder ORM has class methods for nearly any kind of database access/operation you can think of. In this post, I’ll cover some of the available methods for retrieving aggregate-level query results. The examples in this post map out Query Builder method chaining to produce results you would get from a raw MySQL query. Continue reading to see more…
(more…)PHP
CodeIgniter Form Helper library – at a glance
Working with forms is common for anyone in the web development field. Forms are used to capture information for one purpose or another. Be it user login credentials, or customer order details, forms are at the heart of interaction in websites and applications across the Internet. The CodeIgniter PHP framework comes with several built-in helper libraries, that assist in rapid web development. One of them is the form helper. In this post, I’ll cover some of the common methods available from the form helper library…
[Keep reading for more SQL database and PHP/Python-centric content >>>]How to Scrape Sports Data with PHP
Anyone can use web scraping (or web crawling) to collect all sorts of useful data. One of the most valuable ways to put a data scraper to work in this day and age is through collecting sports data. There are many ways to build a web scraper, but we’ll walk you through several ways to do so with PHP programming in this guide.
What Is Data Scraping?
Scraping data from the web involves building a scraper (or “spider”) that can autonomously go to the website you instruct it to, download that website’s HTML, record certain variables, and sometimes even export those valuables into a neat table for you to use later.
In some cases, sophisticated web scraping programs can even upload the information they collect into a database for better organization. While anyone can build a web scraper, this practice is most commonly used in the business world, where competing companies use web crawlers and other programs to keep tabs on their competitors’ products and prices.
Google is one of the best examples of web crawlers we have today. Google uses its algorithms to search web pages for specific keywords and phrases. Then, Google uses these words and phrases to index and categorize the web page based on what it’s about.
For example, if the Google web crawler reads sports, scores, and basketball on your page, in all likelihood, the search engine will categorize your page as a basketball website of some kind.
Before you do any web scraping, however, make sure you’re aware of the privacy policies of any websites you want to scrape. Some websites don’t allow it in their terms of use, so you could potentially get into legal trouble if you’re not careful!
Scraping Sports Data with PHP Programming
There are many different ways to build a data scraper, and doing so with PHP is certainly not the easiest. However, if you have some existing knowledge of HTML and PHP programming, any of these data scraping tools will get the job done. However, if you don’t know much about PHP, you’d probably benefit from learning more about it first through PHP’s syntax guide.
In this section, we’ll look at a few of the best tools available on the internet for web scraping with PHP programming. While you can build your own web scraper from scratch with PHP if you want, doing so is unnecessarily complicated with all of the pre-build resources available online.
While these programs provide an excellent starting point, you’ll still need PHP knowledge to set them up and configure them for sports data scraping. Once you download and explore each program a bit, you should be able to tweak it as needed to extract the data you want.
Simple HTML DOM
A DOM Parser is one of the most flexible ways to parse through HTML, so this Simple HTML DOM is a great place to start when building a web spider. A DOM Parser works by navigating to the website you specify, downloading a page’s HTML, and navigating through it to isolate the tags you specify.
The Simple HTML DOM is excellent because it can work with invalid or messy HTML, while many other PHP-based web crawlers cannot. Because this parser is more or less built for you already, the coding you need to do is minimal.
You can find the full documentation for this DOM Parser here. Keep in mind that this program is written in PHP5+, so you must use PHP5+ or PHP7 with it.
cURL
cURL is a fully-supported, open-source PHP library that is extremely popular among users. cURL works to find and extract data from URLs, so it works great for PHP-based web scraping. As a bonus, since cURL relies only on PHP, you don’t need to download or install any other programs to build a cURL-based web crawler.
cURL stands for “client URL Library,” and it expands PHP’s native capabilities to support HTTP and HTTPS protocols. You can learn how to use cURL and its related functions through its documentation, located here.
Goutte
Goutte (pronounced “goot”) is a PHP-based web scraping program designed to be user-friendly and straightforward, even for those who don’t have much experience with PHP. While Goutte has a few dependencies, if you don’t mind that aspect, it’s one of the easiest ways to set up a simple sports data web crawler through PHP.
Goutte’s documentation can be found right on the home page of its website above, but keep in mind that you’ll need both PHP 5.5+ and Guzzle 6+ to make Goutte work. We’ll look more into Guzzle and what it does below.
Guzzle
Guzzle is a PHP-based HTTP client that lets your program send HTTP requests. It also allows your web crawler to integrate with other internet services. Because Guzzle gives you the ability to send and receive HTTP data, you can use Guzzle alone to create a web crawler if you prefer.
However, because Guzzle is an extremely robust program with many high-level functions, those who don’t have much experience with PHP may struggle to get the most out of it. That’s why companion programs like Goutte generally provide a much more user-friendly place to start with building web crawlers.
Conclusion
Did you see any programs on this list that you’ve used with PHP before? Perhaps you’ve built a web crawler with another programming language, but now you want to test yourself by making one with PHP instead? While PHP can be a challenge to use sometimes, it’s undeniable that building a PHP-based web crawler is easier today than it was years ago because of all the premade options available to you.
Regardless of what sport you end up using your PHP programming web scraper for, it’ll surely boost your productivity and make your sports data gathering much more straightforward. After all, the alternative to building your own web crawler is either copy-and-pasting the information by hand or purchasing a pre-build web crawler for money. Fortunately, all of the options on this list are free, open-source, and ready for you to utilize.
Zenscrape’s Social Media Handles
About the guest contributor, Christoph Leitner
Christoph is a code-loving father of two beautiful children. He is a full-stack developer and a committed team member at Zenscrape.com – a subsidiary of saas.industries. When he isn’t building software, Christoph can be found spending time with his family or training for his next marathon.
PHP strtoupper() and strtolower() string functions
Manipulating text or string data is such a common task that most all programming languages provide built-in functions for this specific purpose. PHP is no different in that regard. The language comes chock-full of different string manipulating functions. If you need string data in all capitalized or upper-cased letters, you can use strtoupper(). What about all lower-case strings? The strtolower() function takes of that. In this post, we visit some rudimentary examples and gain a feel for how these 2 functions operate.
CSV resource for SQL and Data professionals
Most any data professional deals with CSV data at some point in their work-flow. Let’s face it, CSV’s are everywhere and are one of the more common data interchange formats. A common task SQL professionals have is loading CSV data into a database table. Or on the converse, extracting SQL data into a CSV file. I have written several posts about CSV’s with MySQL, PostgreSQL, and the Python pandas library, and still to this day, refer to many of those posts for help when working with CSV’s. This compilation post is for SQL and data professionals who would find value in this type of resource as well.
[Keep reading for more SQL database and PHP/Python-centric content >>>]