01. Python API Syntax
Initialization
First of all, you need to run the following cells to initialize the
-API and authorize your account. You will be prompted to sign-in to the
-account and allow access to View and Manage your Earth Engine
-data. Once you approve, you will get an a verification code that
-needs to be entered at the prompt. This step needs to be done just once
+API and authorize your account. You must have a Google Cloud Project
+associated with your GEE account. Replace the cloud_project
+with your own project from Google Cloud Console.
You will be prompted to allow the notebook to access your Google +credentials to sign-in to the account and allow access to Google +Drive and Google Cloud data. Once you approve, it will proceed to +initialize the Earth Engine API. This step needs to be done just once per session.
- +class="sourceCode python">cloud_project = 'spatialthoughts'
+
+try:
+ ee.Initialize(project=cloud_project)
+except:
+ ee.Authenticate()
+ ee.Initialize(project=cloud_project)Variables
@@ -4575,26 +4577,26 @@Variables
var population = 881549 print(population) - +Earth Engine Objects
You can create Earth Engine objects using the ee
functions the same way.
Line Continuation
@@ -4605,11 +4607,11 @@Line Continuation
var filtered = s2.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) .filter(ee.Filter.date('2019-02-01', '2019-03-01')) .filter(ee.Filter.bounds(geometry)); - +Functions
@@ -4638,24 +4640,24 @@Functions
var ndvi = image.normalizedDifference(['B8', 'B4']).rename('ndvi'); return image.addBands(ndvi); } -def maskS2clouds(image):
- qa = image.select('QA60')
- cloudBitMask = 1 << 10
- cirrusBitMask = 1 << 11
- mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
- qa.bitwiseAnd(cirrusBitMask).eq(0))
- return image.updateMask(mask) \
- .select("B.*") \
- .copyProperties(image, ["system:time_start"])
-
-def addNDVI(image):
- ndvi = image.normalizedDifference(['B8', 'B4']).rename('ndvi')
- return image.addBands(ndvi)
-
-withNdvi = filtered \
- .map(maskS2clouds) \
- .map(addNDVI)def maskS2clouds(image):
+ qa = image.select('QA60')
+ cloudBitMask = 1 << 10
+ cirrusBitMask = 1 << 11
+ mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
+ qa.bitwiseAnd(cirrusBitMask).eq(0))
+ return image.updateMask(mask) \
+ .select("B.*") \
+ .copyProperties(image, ["system:time_start"])
+
+def addNDVI(image):
+ ndvi = image.normalizedDifference(['B8', 'B4']).rename('ndvi')
+ return image.addBands(ndvi)
+
+withNdvi = filtered \
+ .map(maskS2clouds) \
+ .map(addNDVI)Function Arguments
@@ -4672,16 +4674,16 @@Function Arguments
scale: 10, maxPixels: 1e10 }) - +Printing Values
@@ -4692,8 +4694,8 @@Printing Values
object.javascript code:
print(stats.get('ndvi')In-line functions
@@ -4704,10 +4706,10 @@In-line functions
var newList = myList.map(function(number) { return ee.Number(number).pow(2); print(newList); - +Exercise
@@ -4731,10 +4733,10 @@Exercise
print(filtered.size());- +
02. Automatic Conversion of Javascript Code to Python
help you use Earth Engine effectively in Python.
+It comes with a function that can help you translate your javascript earth engine code to Python automatically.
-Google Colab doesn’t come pre-installed with the package, so we -install it via pip.
+The geemap package is pre-installed in Colab.
+
-
+class="sourceCode python">
+Initialization
+First of all, you need to run the following cells to initialize the
+API and authorize your account. You must have a Google Cloud Project
+associated with your GEE account. Replace the cloud_project
+with your own project from Google Cloud Console.
try:
- import geemap
-except ModuleNotFoundError:
- if 'google.colab' in str(get_ipython()):
- print('geemap not found, installing via pip in Google Colab...')
- !pip install geemap --quiet
- import geemap
- else:
- print('geemap not found, please install via conda in your environment')The automatic conversion of code is done by calling the
-geemap.js_snippet_to_py() function. We first create a
-string with the javascript code.
javascript_code = """
-var geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766]);
-Map.centerObject(geometry, 12)
-var s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
-var rgbVis = {
- min: 0.0,
- max: 3000,
- bands: ['B4', 'B3', 'B2'],
-};
-
-// Write a function for Cloud masking
-function maskS2clouds(image) {
- var qa = image.select('QA60')
- var cloudBitMask = 1 << 10;
- var cirrusBitMask = 1 << 11;
- var mask = qa.bitwiseAnd(cloudBitMask).eq(0).and(
- qa.bitwiseAnd(cirrusBitMask).eq(0))
- return image.updateMask(mask)
- .select("B.*")
- .copyProperties(image, ["system:time_start"])
-}
-
-var filtered = s2
- .filter(ee.Filter.date('2019-01-01', '2019-12-31'))
- .filter(ee.Filter.bounds(geometry))
- .map(maskS2clouds)
-
-
-// Write a function that computes NDVI for an image and adds it as a band
-function addNDVI(image) {
- var ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi');
- return image.addBands(ndvi);
-}
-
-var withNdvi = filtered.map(addNDVI);
-
-var composite = withNdvi.median()
-palette = [
- 'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163', '99B718',
- '74A901', '66A000', '529400', '3E8601', '207401', '056201',
- '004C00', '023B01', '012E01', '011D01', '011301'];
-
-ndviVis = {min:0, max:0.5, palette: palette }
-Map.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite')
-
-"""cloud_project = 'spatialthoughts'
+
+try:
+ ee.Initialize(project=cloud_project)
+except:
+ ee.Authenticate()
+ ee.Initialize(project=cloud_project)
+
+class="sourceCode python">
+Automatic Conversion using GUI
+geemap comes with a user interface that can be used to
+interactively do code conversion. Let’s try to convert the following
+Javascript code to Python.
var geometry = ee.Geometry.Point([77.60412933051538, 12.952912912328241]);
+var s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED');
+
+var rgbVis = {min: 0.0, max: 3000, bands: ['B4', 'B3', 'B2']};
+
+var filtered = s2.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30))
+ .filter(ee.Filter.date('2019-01-01', '2020-01-01'))
+ .filter(ee.Filter.bounds(geometry));
+
+var medianComposite = filtered.median();
+
+Map.centerObject(geometry, 10);
+Map.addLayer(medianComposite, rgbVis, 'Median Composite');Run the cell below to load the map widget. Once the map widget loads, +click the Toolbar icon in the top-right corner and select the +Convert Earth Engine Javascript to Python tool. Paste your +Javascript code and click Convert.
lines = geemap.js_snippet_to_py(
- javascript_code, add_new_cell=False,
- import_ee=True, import_geemap=True, show_map=True)
-for line in lines:
- print(line.rstrip())m = geemap.Map(width=800)
+mgeometry = ee.Geometry.Point([77.60412933051538, 12.952912912328241])
+s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
+
+rgbVis = {'min': 0.0, 'max': 3000, 'bands': ['B4', 'B3', 'B2']}
+
+filtered = s2.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) \
+ .filter(ee.Filter.date('2019-01-01', '2020-01-01')) \
+ .filter(ee.Filter.bounds(geometry))
+
+medianComposite = filtered.median()
+
+m.centerObject(geometry, 10)
+m.addLayer(medianComposite, rgbVis, 'Median Composite')You will see the auto-converted code displayed. Copy and paste it
+into a new cell and run it. Your code will be run using the GEE Python
+API. If your code loads any layers, they will be loadded on the map
+widget. To display it, open a new code cell and just type m
+to display the widget.
++ +Note The auto-conversion is almost perfect and works flawlessly on +most GEE code. One place it misses is during the conversion of function +arguments specified as a dicitonary. You will need to prefix the +resulting code with
+**to specify them as +**kwargs. For example, thegeemapconverter +produces code such as below.+stats = image.reduceRegion({ +'reducer': ee.Reducer.mean(), +'geometry': geometry, +'scale': 10, +'maxPixels': 1e10 +})To make this valid GEE Python API code - prefix the argument +dictionary with
+**.+stats = image.reduceRegion(**{ +'reducer': ee.Reducer.mean(), +'geometry': geometry, +'scale': 10, +'maxPixels': 1e10 +})
+
+
+
+
Automatic Conversion using Code
+geemap offers a function js_snippet_to_py()
+that can be used to perform the conversion using code. This is useful
+for batch conversions. To use this, we first create a string with the
+javascript code.
javascript_code = """
+var geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766]);
+Map.centerObject(geometry, 12)
+var s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
+var rgbVis = {
+ min: 0.0,
+ max: 3000,
+ bands: ['B4', 'B3', 'B2'],
+};
+
+// Write a function for Cloud masking
+function maskS2clouds(image) {
+ var qa = image.select('QA60')
+ var cloudBitMask = 1 << 10;
+ var cirrusBitMask = 1 << 11;
+ var mask = qa.bitwiseAnd(cloudBitMask).eq(0).and(
+ qa.bitwiseAnd(cirrusBitMask).eq(0))
+ return image.updateMask(mask)
+ .select("B.*")
+ .copyProperties(image, ["system:time_start"])
+}
+
+var filtered = s2
+ .filter(ee.Filter.date('2019-01-01', '2019-12-31'))
+ .filter(ee.Filter.bounds(geometry))
+ .map(maskS2clouds)
+
+
+// Write a function that computes NDVI for an image and adds it as a band
+function addNDVI(image) {
+ var ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi');
+ return image.addBands(ndvi);
+}
+
+var withNdvi = filtered.map(addNDVI);
+
+var composite = withNdvi.median()
+palette = [
+ 'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163', '99B718',
+ '74A901', '66A000', '529400', '3E8601', '207401', '056201',
+ '004C00', '023B01', '012E01', '011D01', '011301'];
+
+ndviVis = {min:0, max:0.5, palette: palette }
+Map.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite')
+
+"""lines = geemap.js_snippet_to_py(
+ javascript_code, add_new_cell=False,
+ import_ee=True, import_geemap=True, show_map=True)
+for line in lines:
+ print(line.rstrip())The automatic conversion works great. Review it and paste it to the cell below.
-import ee
-import geemap
-Map = geemap.Map()
-
-geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766])
-Map.centerObject(geometry, 12)
-s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
-rgbVis = {
- 'min': 0.0,
- 'max': 3000,
- 'bands': ['B4', 'B3', 'B2'],
-}
-
-# Write a function for Cloud masking
-def maskS2clouds(image):
- qa = image.select('QA60')
- cloudBitMask = 1 << 10
- cirrusBitMask = 1 << 11
- mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
- qa.bitwiseAnd(cirrusBitMask).eq(0))
- return image.updateMask(mask) \
- .select("B.*") \
- .copyProperties(image, ["system:time_start"])
-
-filtered = s2 \
- .filter(ee.Filter.date('2019-01-01', '2019-12-31')) \
- .filter(ee.Filter.bounds(geometry)) \
- .map(maskS2clouds)
-
-# Write a function that computes NDVI for an image and adds it as a band
-def addNDVI(image):
- ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi')
- return image.addBands(ndvi)
-
-withNdvi = filtered.map(addNDVI)
-
-composite = withNdvi.median()
-palette = [
- 'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163', '99B718',
- '74A901', '66A000', '529400', '3E8601', '207401', '056201',
- '004C00', '023B01', '012E01', '011D01', '011301']
-
-ndviVis = {'min':0, 'max':0.5, 'palette': palette }
-Map.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite')
-Mapimport ee
+import geemap
+m = geemap.Map()
+
+geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766])
+m.centerObject(geometry, 12)
+s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
+rgbVis = {
+ 'min': 0.0,
+ 'max': 3000,
+ 'bands': ['B4', 'B3', 'B2'],
+}
+
+# Write a function for Cloud masking
+def maskS2clouds(image):
+ qa = image.select('QA60')
+ cloudBitMask = 1 << 10
+ cirrusBitMask = 1 << 11
+ mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
+ qa.bitwiseAnd(cirrusBitMask).eq(0))
+ return image.updateMask(mask) \
+ .select("B.*") \
+ .copyProperties(image, ["system:time_start"])
+
+filtered = s2 \
+ .filter(ee.Filter.date('2019-01-01', '2019-12-31')) \
+ .filter(ee.Filter.bounds(geometry)) \
+ .map(maskS2clouds)
+
+# Write a function that computes NDVI for an image and adds it as a band
+def addNDVI(image):
+ ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi')
+ return image.addBands(ndvi)
+
+withNdvi = filtered.map(addNDVI)
+
+composite = withNdvi.median()
+palette = [
+ 'FFFFFF', 'CE7E45', 'DF923D', 'F1B555', 'FCD163', '99B718',
+ '74A901', '66A000', '529400', '3E8601', '207401', '056201',
+ '004C00', '023B01', '012E01', '011D01', '011301']
+
+ndviVis = {'min':0, 'max':0.5, 'palette': palette }
+m.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite')
+mExercise
Take the Javascript code snippet below and use geemap to
@@ -4907,92 +4990,104 @@
03. Batch Exports
href="https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FEnd-to-End-GEE%3ASupplement%2FImage_Collections%2FExporting_ImageCollections">recommended script. - - - +
+
Initialization
+First of all, you need to run the following cells to initialize the
+API and authorize your account. You must have a Google Cloud Project
+associated with your GEE account. Replace the cloud_project
+with your own project from Google Cloud Console.
Create a Collection
-geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766])
-s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
-rgbVis = {
- 'min': 0.0,
- 'max': 3000,
- 'bands': ['B4', 'B3', 'B2'],
-}
-
-# Write a function for Cloud masking
-def maskS2clouds(image):
- qa = image.select('QA60')
- cloudBitMask = 1 << 10
- cirrusBitMask = 1 << 11
- mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
- qa.bitwiseAnd(cirrusBitMask).eq(0))
- return image.updateMask(mask) \
- .select("B.*") \
- .copyProperties(image, ["system:time_start"])
-
-filtered = s2 \
- .filter(ee.Filter.date('2019-01-01', '2020-01-01')) \
- .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) \
- .filter(ee.Filter.bounds(geometry)) \
- .map(maskS2clouds)
-
-# Write a function that computes NDVI for an image and adds it as a band
-def addNDVI(image):
- ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi')
- return image.addBands(ndvi)
-
-withNdvi = filtered.map(addNDVI)geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766])
+s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')
+rgbVis = {
+ 'min': 0.0,
+ 'max': 3000,
+ 'bands': ['B4', 'B3', 'B2'],
+}
+
+# Write a function for Cloud masking
+def maskS2clouds(image):
+ qa = image.select('QA60')
+ cloudBitMask = 1 << 10
+ cirrusBitMask = 1 << 11
+ mask = qa.bitwiseAnd(cloudBitMask).eq(0).And(
+ qa.bitwiseAnd(cirrusBitMask).eq(0))
+ return image.updateMask(mask) \
+ .select("B.*") \
+ .copyProperties(image, ["system:time_start"])
+
+filtered = s2 \
+ .filter(ee.Filter.date('2019-01-01', '2020-01-01')) \
+ .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) \
+ .filter(ee.Filter.bounds(geometry)) \
+ .map(maskS2clouds)
+
+# Write a function that computes NDVI for an image and adds it as a band
+def addNDVI(image):
+ ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi')
+ return image.addBands(ndvi)
+
+withNdvi = filtered.map(addNDVI)Export All Images
Exports are done via the ee.batch module. This module
allows you to automatically start an export - making it suitable for
batch exports.
image_ids = withNdvi.aggregate_array('system:index').getInfo()
-print('Total images: ', len(image_ids))# Export with 100m resolution for this demo
-for i, image_id in enumerate(image_ids):
- image = ee.Image(withNdvi.filter(ee.Filter.eq('system:index', image_id)).first())
- task = ee.batch.Export.image.toDrive(**{
- 'image': image.select('ndvi'),
- 'description': 'Image Export {}'.format(i+1),
- 'fileNamePrefix': image_id,
- 'folder':'earthengine',
- 'scale': 100,
- 'region': image.geometry(),
- 'maxPixels': 1e10
- })
- task.start()
- print('Started Task: ', i+1)image_ids = withNdvi.aggregate_array('system:index').getInfo()
+print('Total images: ', len(image_ids))# Export with 100m resolution for this demo
+for i, image_id in enumerate(image_ids):
+ image = ee.Image(withNdvi.filter(ee.Filter.eq('system:index', image_id)).first())
+ task = ee.batch.Export.image.toDrive(**{
+ 'image': image.select('ndvi'),
+ 'description': 'Image Export {}'.format(i+1),
+ 'fileNamePrefix': image_id,
+ 'folder':'earthengine',
+ 'scale': 100,
+ 'region': image.geometry(),
+ 'maxPixels': 1e10
+ })
+ task.start()
+ print('Started Task: ', i+1)Manage Running/Waiting Tasks
You can manage tasks as well. Get a list of tasks and get state information on them
-tasks = ee.batch.Task.list()
-for task in tasks:
- task_id = task.status()['id']
- task_state = task.status()['state']
- print(task_id, task_state)tasks = ee.batch.Task.list()
+for task in tasks:
+ task_id = task.status()['id']
+ task_state = task.status()['state']
+ print(task_id, task_state)You can cancel tasks as well
-tasks = ee.batch.Task.list()
-for task in tasks:
- task_id = task.status()['id']
- task_state = task.status()['state']
- if task_state == 'RUNNING' or task_state == 'READY':
- task.cancel()
- print('Task {} canceled'.format(task_id))
- else:
- print('Task {} state is {}'.format(task_id, task_state))tasks = ee.batch.Task.list()
+for task in tasks:
+ task_id = task.status()['id']
+ task_state = task.status()['state']
+ if task_state == 'RUNNING' or task_state == 'READY':
+ task.cancel()
+ print('Task {} canceled'.format(task_id))
+ else:
+ print('Task {} state is {}'.format(task_id, task_state))Exercise
@@ -5007,38 +5102,38 @@Exercise
import ee
-
-lsib = ee.FeatureCollection('USDOS/LSIB_SIMPLE/2017')
-australia = lsib.filter(ee.Filter.eq('country_na', 'Australia'))
-geometry = australia.geometry()
-
-terraclimate = ee.ImageCollection('IDAHO_EPSCOR/TERRACLIMATE')
-tmax = terraclimate.select('tmmx')
-
-def scale(image):
- return image.multiply(0.1) \
- .copyProperties(image,['system:time_start'])
-
-tmaxScaled = tmax.map(scale)
-
-filtered = tmaxScaled \
- .filter(ee.Filter.date('2020-01-01', '2021-01-01')) \
- .filter(ee.Filter.bounds(geometry))
-
-image_ids = filtered.aggregate_array('system:index').getInfo()
-print('Total images: ', len(image_ids))import ee
+
+lsib = ee.FeatureCollection('USDOS/LSIB_SIMPLE/2017')
+australia = lsib.filter(ee.Filter.eq('country_na', 'Australia'))
+geometry = australia.geometry()
+
+terraclimate = ee.ImageCollection('IDAHO_EPSCOR/TERRACLIMATE')
+tmax = terraclimate.select('tmmx')
+
+def scale(image):
+ return image.multiply(0.1) \
+ .copyProperties(image,['system:time_start'])
+
+tmaxScaled = tmax.map(scale)
+
+filtered = tmaxScaled \
+ .filter(ee.Filter.date('2020-01-01', '2021-01-01')) \
+ .filter(ee.Filter.bounds(geometry))
+
+image_ids = filtered.aggregate_array('system:index').getInfo()
+print('Total images: ', len(image_ids))Replace the comments with your code.
-for i, image_id in enumerate(image_ids):
- exportImage = ee.Image(filtered.filter(ee.Filter.eq('system:index', image_id)).first())
- # Clip the image to the region geometry
- clippedImage = exportImage.clip(geometry)
-
- ## Create the export task using ee.batch.Export.image.toDrive()
-
- ## Start the taskfor i, image_id in enumerate(image_ids):
+ exportImage = ee.Image(filtered.filter(ee.Filter.eq('system:index', image_id)).first())
+ # Clip the image to the region geometry
+ clippedImage = exportImage.clip(geometry)
+
+ ## Create the export task using ee.batch.Export.image.toDrive()
+
+ ## Start the task
@@ -5064,100 +5159,106 @@
04. Creating Charts in Python
try:
- import geemap
-except ModuleNotFoundError:
- if 'google.colab' in str(get_ipython()):
- print('geemap not found, installing via pip in Google Colab...')
- !pip install geemap --quiet
- import geemap
- else:
- print('geemap not found, please install via conda in your environment')Load the TerraClimate collection and select the ‘tmmx’ band.
- -Define a point location for the chart.
+
+
Initialization
+First of all, you need to run the following cells to initialize the
+API and authorize your account. You must have a Google Cloud Project
+associated with your GEE account. Replace the cloud_project
+with your own project from Google Cloud Console.
Scale the band values so they are in Degree Celcius.
+class="sourceCode python">import geemap
+import eedef scale_image(image):
- return ee.Image(image).multiply(0.1)\
- .copyProperties(image, ['system:time_start'])
-
-tmaxScaled = tmax.map(scale_image)Filter the collection.
+class="sourceCode python">cloud_project = 'spatialthoughts'
+
+try:
+ ee.Initialize(project=cloud_project)
+except:
+ ee.Authenticate()
+ ee.Initialize(project=cloud_project)
+
-
+Load and Process the Data
+Load the TerraClimate collection and select the ‘tmmx’ band.
filtered = tmaxScaled.filter(ee.Filter.date('2019-01-01', '2020-01-01')) \
- .filter(ee.Filter.bounds(geometry))To chart an image series in Python, we must first extract the values -from each image and create a FeatureCollection.
+class="sourceCode python">terraclimate = ee.ImageCollection('IDAHO_EPSCOR/TERRACLIMATE')
+tmax = terraclimate.select('tmmx')Define a point location for the chart.
def extract_data(image):
- stats = image.reduceRegion(**{
- 'reducer':ee.Reducer.mean(),
- 'geometry':geometry,
- 'scale':5000
- })
- properties = {
- 'month': image.get('system:index'),
- 'tmmx': stats.get('tmmx')
- }
- return ee.Feature(None, properties)
-
-data = ee.FeatureCollection(filtered.map(extract_data))geometry = ee.Geometry.Point([77.57738128916243, 12.964758918835752])Scale the band values so they are in Degree Celcius.
-
-
+Create an Interactive Chart using geemap
+class="sourceCode python">def scale_image(image):
+ return ee.Image(image).multiply(0.1)\
+ .copyProperties(image, ['system:time_start'])
+
+tmaxScaled = tmax.map(scale_image)Filter the collection.
+class="sourceCode python">filtered = tmaxScaled.filter(ee.Filter.date('2019-01-01', '2020-01-01')) \
+ .filter(ee.Filter.bounds(geometry))
+To chart an image series in Python, we must first extract the values +from each image and create a FeatureCollection.
options = {
- 'title': 'Max Monthly Temperature at Bangalore',
- 'legend_location': 'top-right',
- 'height': '500px',
- 'ylabel': 'Temperature (C)',
- 'xlabel': 'Date',
- 'colors': ['blue']
-}def extract_data(image):
+ stats = image.reduceRegion(**{
+ 'reducer':ee.Reducer.mean(),
+ 'geometry':geometry,
+ 'scale':5000
+ })
+ properties = {
+ 'month': image.get('system:index'),
+ 'tmmx': stats.get('tmmx')
+ }
+ return ee.Feature(None, properties)
+
+data = ee.FeatureCollection(filtered.map(extract_data))
+class="sourceCode python">print(data.first().getInfo())
-
-
+class="sourceCode python">Create a chart using Matplotlib
-We can convert a FeatureCollection to a DataFrame using
-geemap helper function ee_to_pandas.
Now we have a regular Pandas dataframe that can be plotted with
-matplotlib.
options = {
+ 'title': 'Max Monthly Temperature at Bangalore',
+ 'legend_location': 'top-right',
+ 'height': '500px',
+ 'ylabel': 'Temperature (C)',
+ 'xlabel': 'Date',
+ 'colors': ['blue']
+}chart.feature_byFeature(data, 'month', ['tmmx'], **options)
+
+
+
+class="sourceCode python">
+
+Create a chart using Matplotlib
+We can convert a FeatureCollection to a DataFrame using
+geemap helper function ee_to_pandas.
fig, ax = plt.subplots()
-fig.set_size_inches(20,10)
-
-
-df.plot(ax=ax,
- title='Max Monthly Temperature at Bangalore',
- x='month',
- ylabel='Temperature (C)',
- kind='line')
-plt.tight_layout()import geemap
+df = geemap.ee_to_pandas(data)Now we have a regular Pandas dataframe that can be plotted with
+matplotlib.
fig, ax = plt.subplots()
+fig.set_size_inches(20,10)
+
+
+df.plot(ax=ax,
+ title='Max Monthly Temperature at Bangalore',
+ x='month',
+ ylabel='Temperature (C)',
+ kind='line')
+plt.tight_layout()Exercise
@@ -5193,68 +5294,74 @@05. Automating Downloads
download_data.py with the
content shown below.import datetime
-import ee
-import csv
-import os
-
-ee.Initialize()
-
-# Get current date and convert to milliseconds
-start_date = ee.Date.fromYMD(2022, 1, 1)
-end_date = start_date.advance(1, 'month')
-
-date_string = end_date.format('YYYY_MM')
-filename = 'ssm_{}.csv'.format(date_string.getInfo())
-
-# Saving to current directory. You can change the path to appropriate location
-output_path = os.path.join(filename)
-
-# Datasets
-# SMAP is in safe mode and not generating new data since August 2022
-# https://nsidc.org/data/user-resources/data-announcements/user-notice-smap-safe-mode
-soilmoisture = ee.ImageCollection("NASA_USDA/HSL/SMAP10KM_soil_moisture")
-admin2 = ee.FeatureCollection("FAO/GAUL_SIMPLIFIED_500m/2015/level2")
-
-# Filter to a state
-karnataka = admin2.filter(ee.Filter.eq('ADM1_NAME', 'Karnataka'))
-
-# Select the ssm band
-ssm = soilmoisture.select('ssm')
-
-filtered = ssm .filter(ee.Filter.date(start_date, end_date))
-
-mean = filtered.mean()
-
-stats = mean.reduceRegions(**{
- 'collection': karnataka,
- 'reducer': ee.Reducer.mean().setOutputs(['meanssm']),
- 'scale': 10000,
- 'crs': 'EPSG:32643'
- })
-
-# Select columns to keep and remove geometry to make the result lightweight
-# Change column names to match your uploaded shapefile
-columns = ['ADM2_NAME', 'meanssm']
-exportCollection = stats.select(**{
- 'propertySelectors': columns,
- 'retainGeometry': False})
-
-features = exportCollection.getInfo()['features']
-
-data = []
-
-for f in features:
- data.append(f['properties'])
-
-field_names = ['ADM2_NAME', 'meanssm']
-
-with open(output_path, 'w') as csvfile:
- writer = csv.DictWriter(csvfile, fieldnames = field_names)
- writer.writeheader()
- writer.writerows(data)
- print('Success: File written at', output_path)import datetime
+import ee
+import csv
+import os
+
+cloud_project = 'spatialthoughts'
+
+try:
+ ee.Initialize(project=cloud_project)
+except:
+ ee.Authenticate()
+ ee.Initialize(project=cloud_project)
+
+# Get current date and convert to milliseconds
+start_date = ee.Date.fromYMD(2022, 1, 1)
+end_date = start_date.advance(1, 'month')
+
+date_string = end_date.format('YYYY_MM')
+filename = 'ssm_{}.csv'.format(date_string.getInfo())
+
+# Saving to current directory. You can change the path to appropriate location
+output_path = os.path.join(filename)
+
+# Datasets
+# SMAP is in safe mode and not generating new data since August 2022
+# https://nsidc.org/data/user-resources/data-announcements/user-notice-smap-safe-mode
+soilmoisture = ee.ImageCollection("NASA_USDA/HSL/SMAP10KM_soil_moisture")
+admin2 = ee.FeatureCollection("FAO/GAUL_SIMPLIFIED_500m/2015/level2")
+
+# Filter to a state
+karnataka = admin2.filter(ee.Filter.eq('ADM1_NAME', 'Karnataka'))
+
+# Select the ssm band
+ssm = soilmoisture.select('ssm')
+
+filtered = ssm .filter(ee.Filter.date(start_date, end_date))
+
+mean = filtered.mean()
+
+stats = mean.reduceRegions(**{
+ 'collection': karnataka,
+ 'reducer': ee.Reducer.mean().setOutputs(['meanssm']),
+ 'scale': 10000,
+ 'crs': 'EPSG:32643'
+ })
+
+# Select columns to keep and remove geometry to make the result lightweight
+# Change column names to match your uploaded shapefile
+columns = ['ADM2_NAME', 'meanssm']
+exportCollection = stats.select(**{
+ 'propertySelectors': columns,
+ 'retainGeometry': False})
+
+features = exportCollection.getInfo()['features']
+
+data = []
+
+for f in features:
+ data.append(f['properties'])
+
+field_names = ['ADM2_NAME', 'meanssm']
+
+with open(output_path, 'w') as csvfile:
+ writer = csv.DictWriter(csvfile, fieldnames = field_names)
+ writer.writeheader()
+ writer.writerows(data)
+ print('Success: File written at', output_path)- From the terminal, navigate to the directory where you have created the file and type the command below to run the script. diff --git a/end-to-end-gee-output/01_python_api_syntax.md b/end-to-end-gee-output/01_python_api_syntax.md index f8eeee9f..eef16816 100644 --- a/end-to-end-gee-output/01_python_api_syntax.md +++ b/end-to-end-gee-output/01_python_api_syntax.md @@ -4,7 +4,9 @@ Coming from the programming in Earth Engine through the Code Editor, you will ne #### Initialization -First of all, you need to run the following cells to initialize the API and authorize your account. You will be prompted to sign-in to the account and allow access to *View and Manage your Earth Engine data*. Once you approve, you will get an a verification code that needs to be entered at the prompt. This step needs to be done just once per session. +First of all, you need to run the following cells to initialize the API and authorize your account. You must have a Google Cloud Project associated with your GEE account. Replace the `cloud_project` with your own project from [Google Cloud Console](https://console.cloud.google.com/). + +You will be prompted to allow the notebook to access your Google credentials to sign-in to the account and allow access to *Google Drive and Google Cloud data*. Once you approve, it will proceed to initialize the Earth Engine API. This step needs to be done just once per session. ```python @@ -13,12 +15,14 @@ import ee ```python -ee.Authenticate() -``` +cloud_project = 'spatialthoughts' +try: + ee.Initialize(project=cloud_project) +except: + ee.Authenticate() + ee.Initialize(project=cloud_project) -```python -ee.Initialize() ``` #### Variables diff --git a/end-to-end-gee-output/02_automatic_conversion_of_scripts.md b/end-to-end-gee-output/02_automatic_conversion_of_scripts.md index 6037a5d9..8597bfc0 100644 --- a/end-to-end-gee-output/02_automatic_conversion_of_scripts.md +++ b/end-to-end-gee-output/02_automatic_conversion_of_scripts.md @@ -1,23 +1,107 @@ -[geemap](https://github.com/giswqs/geemap) is an open-source Python package that comes with many helpful features that help you use Earth Engine effectively in Python. +[geemap](https://github.com/giswqs/geemap) is an open-source Python package that comes with many helpful features that help you use Earth Engine effectively in Python. It comes with a function that can help you translate your javascript earth engine code to Python automatically. -Google Colab doesn't come pre-installed with the package, so we install it via pip. +The `geemap` package is pre-installed in Colab. ```python +import geemap +import ee +``` + +#### Initialization + +First of all, you need to run the following cells to initialize the API and authorize your account. You must have a Google Cloud Project associated with your GEE account. Replace the `cloud_project` with your own project from [Google Cloud Console](https://console.cloud.google.com/). + + +```python +cloud_project = 'spatialthoughts' + try: - import geemap -except ModuleNotFoundError: - if 'google.colab' in str(get_ipython()): - print('geemap not found, installing via pip in Google Colab...') - !pip install geemap --quiet - import geemap - else: - print('geemap not found, please install via conda in your environment') + ee.Initialize(project=cloud_project) +except: + ee.Authenticate() + ee.Initialize(project=cloud_project) +``` + +#### Automatic Conversion using GUI + +`geemap` comes with a user interface that can be used to interactively do code conversion. Let's try to convert the following Javascript code to Python. + +``` +var geometry = ee.Geometry.Point([77.60412933051538, 12.952912912328241]); +var s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED'); + +var rgbVis = {min: 0.0, max: 3000, bands: ['B4', 'B3', 'B2']}; + +var filtered = s2.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) + .filter(ee.Filter.date('2019-01-01', '2020-01-01')) + .filter(ee.Filter.bounds(geometry)); + +var medianComposite = filtered.median(); + +Map.centerObject(geometry, 10); +Map.addLayer(medianComposite, rgbVis, 'Median Composite'); +``` + +Run the cell below to load the map widget. Once the map widget loads, click the *Toolbar* icon in the top-right corner and select the *Convert Earth Engine Javascript to Python* tool. Paste your Javascript code and click *Convert*. + + +```python +m = geemap.Map(width=800) +m ``` -The automatic conversion of code is done by calling the `geemap.js_snippet_to_py()` function. We first create a string with the javascript code. + +```python +geometry = ee.Geometry.Point([77.60412933051538, 12.952912912328241]) +s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED') + +rgbVis = {'min': 0.0, 'max': 3000, 'bands': ['B4', 'B3', 'B2']} + +filtered = s2.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) \ + .filter(ee.Filter.date('2019-01-01', '2020-01-01')) \ + .filter(ee.Filter.bounds(geometry)) + +medianComposite = filtered.median() + +m.centerObject(geometry, 10) +m.addLayer(medianComposite, rgbVis, 'Median Composite') + +``` + +You will see the auto-converted code displayed. Copy and paste it into a new cell and run it. Your code will be run using the GEE Python API. If your code loads any layers, they will be loadded on the map widget. To display it, open a new code cell and just type `m` to display the widget. + + + + +```python +m +``` + +> Note The auto-conversion is almost perfect and works flawlessly on most GEE code. One place it misses is during the conversion of function arguments specified as a dicitonary. You will need to prefix the resulting code with `**` to specify them as `**kwargs`. For example, the `geemap` converter produces code such as below. + ``` + stats = image.reduceRegion({ + 'reducer': ee.Reducer.mean(), + 'geometry': geometry, + 'scale': 10, + 'maxPixels': 1e10 + }) + ``` +To make this valid GEE Python API code - prefix the argument dictionary with `**`. + ``` + stats = image.reduceRegion(**{ + 'reducer': ee.Reducer.mean(), + 'geometry': geometry, + 'scale': 10, + 'maxPixels': 1e10 + }) + ``` + +#### Automatic Conversion using Code + +`geemap` offers a function `js_snippet_to_py()` that can be used to perform the conversion using code. This is useful for batch conversions. To use this, we first create a string with the javascript code. ```python @@ -42,12 +126,12 @@ function maskS2clouds(image) { .select("B.*") .copyProperties(image, ["system:time_start"]) } - + var filtered = s2 .filter(ee.Filter.date('2019-01-01', '2019-12-31')) .filter(ee.Filter.bounds(geometry)) .map(maskS2clouds) - + // Write a function that computes NDVI for an image and adds it as a band function addNDVI(image) { @@ -84,10 +168,10 @@ The automatic conversion works great. Review it and paste it to the cell below. ```python import ee import geemap -Map = geemap.Map() +m = geemap.Map() geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766]) -Map.centerObject(geometry, 12) +m.centerObject(geometry, 12) s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED') rgbVis = { 'min': 0.0, @@ -125,8 +209,8 @@ palette = [ '004C00', '023B01', '012E01', '011D01', '011301'] ndviVis = {'min':0, 'max':0.5, 'palette': palette } -Map.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite') -Map +m.addLayer(withNdvi.select('ndvi'), ndviVis, 'NDVI Composite') +m ``` ### Exercise diff --git a/end-to-end-gee-output/03_export_a_collection.md b/end-to-end-gee-output/03_export_a_collection.md index 1efa3c4d..29b92c66 100644 --- a/end-to-end-gee-output/03_export_a_collection.md +++ b/end-to-end-gee-output/03_export_a_collection.md @@ -2,19 +2,24 @@ One of the most commonly asked questions by Earth Engine users is - *How do I do > You can also export images in a collection using Javascript API in the Code Editor but this requires you to manually start the tasks for each image. This approach is fine for small number of images. You can check out the [recommended script](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FEnd-to-End-GEE%3ASupplement%2FImage_Collections%2FExporting_ImageCollections). +#### Initialization -```python -import ee -``` +First of all, you need to run the following cells to initialize the API and authorize your account. You must have a Google Cloud Project associated with your GEE account. Replace the `cloud_project` with your own project from [Google Cloud Console](https://console.cloud.google.com/). ```python -ee.Authenticate() +import ee ``` ```python -ee.Initialize() +cloud_project = 'spatialthoughts' + +try: + ee.Initialize(project=cloud_project) +except: + ee.Authenticate() + ee.Initialize(project=cloud_project) ``` #### Create a Collection diff --git a/end-to-end-gee-output/04_creating_charts.md b/end-to-end-gee-output/04_creating_charts.md index b69f4ea3..a2a9b46f 100644 --- a/end-to-end-gee-output/04_creating_charts.md +++ b/end-to-end-gee-output/04_creating_charts.md @@ -8,34 +8,29 @@ References: - geemap [Example notebook](https://geemap.org/notebooks/63_charts/) +#### Initialization -```python -import ee -``` +First of all, you need to run the following cells to initialize the API and authorize your account. You must have a Google Cloud Project associated with your GEE account. Replace the `cloud_project` with your own project from [Google Cloud Console](https://console.cloud.google.com/). ```python -try: - import geemap -except ModuleNotFoundError: - if 'google.colab' in str(get_ipython()): - print('geemap not found, installing via pip in Google Colab...') - !pip install geemap --quiet - import geemap - else: - print('geemap not found, please install via conda in your environment') +import geemap +import ee ``` ```python -ee.Authenticate() -``` - +cloud_project = 'spatialthoughts' -```python -ee.Initialize() +try: + ee.Initialize(project=cloud_project) +except: + ee.Authenticate() + ee.Initialize(project=cloud_project) ``` +#### Load and Process the Data + Load the TerraClimate collection and select the 'tmmx' band. @@ -75,7 +70,7 @@ To chart an image series in Python, we must first extract the values from each i ```python def extract_data(image): - stats = image.reduceRegion(**{ + stats = image.reduceRegion(**{ 'reducer':ee.Reducer.mean(), 'geometry':geometry, 'scale':5000 @@ -94,7 +89,7 @@ data = ee.FeatureCollection(filtered.map(extract_data)) print(data.first().getInfo()) ``` -### Create an Interactive Chart using geemap +#### Create an Interactive Chart using geemap ```python @@ -104,7 +99,7 @@ from geemap import chart ```python options = { - 'title': 'Max Monthly Temperature at Bangalore', + 'title': 'Max Monthly Temperature at Bangalore', 'legend_location': 'top-right', 'height': '500px', 'ylabel': 'Temperature (C)', @@ -118,7 +113,7 @@ options = { chart.feature_byFeature(data, 'month', ['tmmx'], **options) ``` -### Create a chart using Matplotlib +#### Create a chart using Matplotlib We can convert a FeatureCollection to a DataFrame using `geemap` helper function `ee_to_pandas`.