Write pandas dataframe into Google bigQuery:

https://pandas.pydata.org/pandas-docs/version/0.21/generated/pandas.DataFrame.to_gbq.html#pandas-dataframe-to-gbq

#pandas #bg #bigquery

How to query on Google Big Query?

Big Query or for short BQ is a data warehousing solution that puts your data on a serverless platform with no limit on data size and processing power. It is design for this specific purpose and returns aggregated results in a fraction of a second most of the time.


In python you can use its library by installing google-cloud-bigquery:

pip install --upgrade google-cloud-bigquery

Now create a service account as stated in link below:

https://cloud.google.com/bigquery/docs/reference/libraries

We assume that you have done the installation and configuration process for now. To query on BQ:

from google.cloud import bigquery
 bigquery_client = bigquery.Client()
 dataset_id = 'MY_PROJECT'
 query = ("SELECT user_id FROM MY_PROJECT.users LIMIT 100")

 # API CALL
 query_job = bigquery_client.query(query)

 for row in query_job:
     print row.user_id, row['user_id']

You can send update commands like the above query too.

#google #BI #bigdata #bigquery #warehouse #data_warehouse

https://googlecloudplatform.github.io/google-cloud-python/latest/bigquery/usage.html#queries

#reference #google #bigquery