csv

CSV to JSON with Python for MySQL upload

Admittedly, I have jumped head-first into the world of JSON within MySQL. I also have a thing for Python, of which I find myself using a lot of in my day job as Pipeline Survey Data Analyst. CSV’s are everywhere: in data at work, on the web, even in my personal life (as you’ll see). Loading CSV data into MySQL is nothing new to anyone working with either of the technologies. As a matter of fact, there are several ways you can get your CSV data into MySQL. I wrote the post, Pyodbc SQL CRUD – Create: Examples with MySQL, you can read where I cover loading CSV data using the Python pyodbc module. But, how about type-casting CSV data – which typically are strings – to a compatible JSON data type? Then push the JSON data to a MySQL JSON column? And that my friends, is the focus of this post. Turns out, Python’s csv and json standard libraries makes the whole process virtually painless. Keep reading and see a simple script I devised…

[Keep reading for more SQL database and Python-centric content >>>]
Advertisements

Pandas to SQL – importing CSV data files into PostgreSQL

My goal with this post is to cover what I have learned while inserting pandas DataFrame values into a PostgreSQL table using SQLAlchemy. Interested in learning about this yourself? Want to see a simple example? You are in the right place so keep reading and learn with me…

[Head this way for great PostgresSQL blogging >>>]

Two handy examples of the psql \copy meta-command.

In this blog post, I will cover example uses of the psql \copy meta-command, for client-side work with the PostgreSQL database server. This command allows users who do not have Superuser privileges, to perform COPY type SQL commands.

(more…)

Wholesale inserts with Python, Psycopg2 and PostgreSQL.

Triple P threat to all data files!!! Hahahaha… Now that I have myself collected and together, in this blog post I will utilize Python, Psycopg2, and PostgreSQL for a bulk upload, importing data from a CSV file to a database table. (more…)

Data prepping with Sed for bulk uploads using the PostgreSQL COPY command.

This post will highlight usage of the PostgreSQL COPY command. Along the way, I will use Sed, a Unix command-line utility, to ‘massage’ sample data into a more uniformed structure. Once the data has been formatted to our requirement, we will then load it into a PostgreSQL table.

(more…)