CSV or SQL? SQL or CSV? How about together? Without a doubt, CSV’s are one of the most common and familiar data interchange formats. Importing and exporting CSV data into or out of an SQL database is a staple process in most every data workflow. I’ve written numerous blog posts on both importing and exporting CSV data in an SQL environment. In this post, I am including all of these specific posts (as of the time of writing) in one easy-to-read and centralized location…
(more…)csv
CSV to JSON with Python for MySQL upload
Admittedly, I have jumped head-first into the world of JSON within MySQL. I also have a thing for Python, of which I find myself using a lot of in my day job as Pipeline Survey Data Analyst. CSV’s are everywhere: in data at work, on the web, even in my personal life (as you’ll see). Loading CSV data into MySQL is nothing new to anyone working with either of the technologies. As a matter of fact, there are several ways you can get your CSV data into MySQL. I wrote the post, Pyodbc SQL CRUD – Create: Examples with MySQL, you can read where I cover loading CSV data using the Python pyodbc module. But, how about type-casting CSV data – which typically are strings – to a compatible JSON data type? Then push the JSON data to a MySQL JSON column? And that my friends, is the focus of this post. Turns out, Python’s csv and json standard libraries makes the whole process virtually painless. Keep reading and see a simple script I devised…
Pandas to SQL – importing CSV data files into PostgreSQL
My goal with this post is to cover what I have learned while inserting pandas DataFrame values into a PostgreSQL table using SQLAlchemy. Interested in learning about this yourself? Want to see a simple example? You are in the right place so keep reading and learn with me…
[Head this way for great PostgresSQL blogging >>>]Two handy examples of the psql \copy meta-command.
In this blog post, I will cover example uses of the psql \copy meta-command, for client-side work with the PostgreSQL database server. This command allows users who do not have Superuser privileges, to perform COPY type SQL commands.
Wholesale inserts with Python, Psycopg2 and PostgreSQL.
Triple P threat to all data files!!! Hahahaha… Now that I have myself collected and together, in this blog post I will utilize Python, Psycopg2, and PostgreSQL for a bulk upload, importing data from a CSV file to a database table. (more…)