How I mastered the lambda from basics mastering the tricky zip dependency error from windows for certain libraries
I came from GCP where using cloud function is as easy as ordering a pizza from restaurant with required amount of cheese and toppings.
But in AWS it was bit trickier so when we have to use python preinstalled dependencies it is fine but for external dependencies it takes a bridge to be constructed (how’s this one in regard with the pizza analogy…just kidding).
HOW I USED THE EXTERNAL LIBRARIES
- I created a virtual environment in windows using below commands.
=>python3 -m venv myVirtualEnv
=>myVirtualEnv\scripts\activate
- Called the pip warrior to to install my dependencies
- Then created a requirements.txt as below:
=> pip freeze > requirements. txt
Created a zip of all the files from below folder (and the lambda_function )in my case.
myVirtualEnv/env/lib/python3.7/site-packages
- Uploaded the zip to s3 bucket of my choice
- In the lambda functions tried to pull in the code
- And hit enter to runnnnnnnnn.
But wait what it dint work as my code has dependency to certain library which didnt support windows as the lambda is based on linux so the dependency needs to be zipped from linux instance.(I am like holy cow at this moment)
CREATING A ZIP FROM LINUX INSTANCE
Hoping that you can connect to a linux instance using mobaxterm.exe using the pem or ppk file, I am moving forward( though i didn’t knew how to use mobaxterm.exe)
- I created a virtual environment in Linux using below commands.
=>python3 -m venv myVirtualEnv/env
=>source ~/myVirtualEnv/env/bin/activate
- Called the pip warrior to to install my dependencies
- Then created a requirements.txt as below:
=> pip freeze > requirements. txt
Created a zip of all the files from below folder (and the lambda_function ) in my case.
=>myVirtualEnv/env/lib/python3.7/site-packages
like this:
=>zip -r9 ${OLDPWD}/name_fo_zip_file.zip .
Noyte: OLDPWD create the zip in the directory you were present before moving to the site-packages folder
- Uploaded the zip to s3 bucket of my choice
- In the lambda functions tried to pull in the code
- And hit enter to runnnnnnnnn.
- And guess waht it ran successfully.
A NEW PROBLEM CAME INTO PICTURE
As i was developing lot of lambda codes so for each code i needed to do the zip from linux instance and also upload it to the s3.
Note: Once you upload the zip you can’t edit inline editor.
So i did the repeat and succeed task again and again. But soon i got a post which said to learn about the lambda layers and i got curious to know about that.
A NEW LAYER CALLED LAMBDA LAYER
Lambda layer came as a saviour to our souls so that we can order the pizza whenever we want to, but how
Steps
- Remember we created the zip of dependencies with our lambda_function in the linux instance
- Create one more final zip with all dependencies (without the lambda function) and upload into the lambda layer
- And then what
- Use the same layer for other lambdas you use and also run the great inline editor of AWS lamda.
That’s it man , we are done.
