Afrikaans
Akan
Albanian
Amharic
Arabic
Armenian
Azerbaijani
Basque
Belarusian
Bemba
Bengali
Bihari
Bosnian
Breton
Bulgarian
Cambodian
Catalan
Cebuano
Cherokee
Chichewa
Chinese (Simplified)
Chinese (Traditional)
Corsican
Croatian
Czech
Danish
Dutch
English
Esperanto
Estonian
Ewe
Faroese
Filipino
Finnish
French
Frisian
Ga
Galician
Georgian
German
Greek
Guarani
Gujarati
Haitian Creole
Hausa
Hawaiian
Hebrew
Hindi
Hmong
Hungarian
Icelandic
Igbo
Indonesian
Interlingua
Irish
Italian
Japanese
Javanese
Kannada
Kazakh
Kinyarwanda
Kirundi
Kongo
Korean
Krio (Sierra Leone)
Kurdish
Kurdish (Soranî)
Kyrgyz
Laothian
Latin
Latvian
Lingala
Lithuanian
Lozi
Luganda
Luo
Luxembourgish
Macedonian
Malagasy
Malay
Malayalam
Maltese
Maori
Marathi
Mauritian Creole
Moldavian
Mongolian
Myanmar (Burmese)
Montenegrin
Nepali
Nigerian Pidgin
Northern Sotho
Norwegian
Norwegian (Nynorsk)
Occitan
Oriya
Oromo
Pashto
Persian
Polish
Portuguese (Brazil)
Portuguese (Portugal)
Punjabi
Quechua
Romanian
Romansh
Runyakitara
Russian
Samoan
Scots Gaelic
Serbian
Serbo-Croatian
Sesotho
Setswana
Seychellois Creole
Shona
Sindhi
Sinhalese
Slovak
Slovenian
Somali
Spanish
Spanish (Latin American)
Sundanese
Swahili
Swedish
Tajik
Tamil
Tatar
Telugu
Thai
Tigrinya
Tonga
Tshiluba
Tumbuka
Turkish
Turkmen
Twi
Uighur
Ukrainian
Urdu
Uzbek
Welsh
Wolof
Xhosa
Yiddish
Yoruba
Zulu
Okay, so in this lecture
we're going to install the CloudWatch unified agent
to collect metrics and log for Amazon EC2 instances.
So to do so, very simple
we're first going to have to create an IAM role.
And this role will allow us to send metrics
and logs to CloudWatch.
So to do so let's go into roles, let's create a role
and this is going to be a role for AWS service
which is an EC2 instance.
So we need to create an EC2 instance role
and then we click on next.
So EC2, just select EC2 here.
Next, we need to find permission policies
to apply to this role.
So for this I'm going to type CloudWatchAgent
and we have two
we have the Server Policy and the Admin Policy.
The one we're going to use right now
is the CloudWatchAgentServerPolicy.
And the reason we do so is that
we want to be able to put metric data into CloudWatch
also to send logs
and finally to get parameters from SSM
because it is possible for us to retrieve a CloudWatch
unified agent configuration directly from SSM
as we will see in this lecture.
So this is enough. Let's click on next.
And then the role name should be, for example
Amazon EC2 role for CloudWatch.
Okay. Let's create this role.
And we're done.
So next what I have to do is
to create an EC2 instance
that will be leveraging this role.
So to do so let's go into the management console
into the EC2 console.
And then I'm going to create an EC2 instance.
So I'll go into launch instance and we'll configure it.
So we're going to use Amazon Linux 2 AMI
and then I will use t2.micro.
We don't need to select a key pair
if we don't want to, which is fine.
We'll use EC2 instance connect
to connect to our EC2 instance.
Then we need to create a new security group
and we'll allow SSH and HTTP traffic
the third one, from the internet
because we'll be launching a web server
and we'll get the log from that server.
So we allow SSH to SSH to our instance
and HTTP to allow port 80 traffic on to our EC2 instance.
This is good.
And then we're good to go.
Let's launch this instance.
Okay. So let's click on it
and we're gonna find the instance right now.
So let me wait for it to be ready.
Okay. So our EC2 instance is now created.
Let's connect to it
using the EC2 instance user connect interface.
So we establish a connection directly into the instance
and now I'm able to issue a few commands.
So the first one I'm going to do
is to elevate my privileges.
So I will do sudo su and now I am a super user
on my instance.
And I will do yum install HTTPD.
This will install a web server and that web server
I say yes.
That web server is going to have traffic
coming from the internet.
And we'll take the log out of HTTP
to connect to our instance
to send that to CloudWatch logs
So the HTTPD service is installed.
And now we're just going to echo "hello world"
into var/www/html/index.html.
So now we have an index HTML file created.
And we need to just start our HTTPD server.
So for this, we'll do sudo systemctl start httpd
This will effectively start our HTTPD service.
And if wanted to allow also it to survive through restarts
we will enable it.
So now back into here, this is the public IP
of my EC2 instance, which is great.
And so if I visit it
now I get the prompt hello world.
So that means that we are indeed able
to access our EC2 instance using
this Apache web server.
And now what I want to show you is that
out of this, this created some log entries.
So if we go into the file
so I'll do cat /var/log/httpd
and then we have access underscore log.
This contains the request log of all
the request made into our server.
As you can see, we can find the fact that
yes we did do a get right now.
So also, should there be any errors
we can do /error_log
and we'll get a list of all the errors
that are happening for our HTTPD server.
So these two logs, the error log
as well as the access log
are the logs that I want to send into CloudWatch logs
using the CloudWatch unify agents.
So to do so, we can follow documentation
but this is quite complicated.
So it's to collect metrics and logs
for Amazon EC2 instances
and I've summarized all the instructions in the code.
So the first thing we have to do is to install the agents
on Amazon Linux 2
and for this we can just run this command
a sudo yum install of the Amazon CloudWatch agent.
So, I say, yes and it's going to be installed.
The next thing we have to do is to run a wizard.
And this is to configure the CloudWatch agent.
This is something we have to do once
and then we'll store the configuration
in the SSM parameter restore.
And it will be retrieved at run time if we need to.
So let's go ahead and clear this
and we're going to run the wizard.
So now it says
on which OS are you planning to use the agent wave?
So Linux.
Then is this EC2 or On-Premises?
This is EC2.
We're going to run this as the route user.
And do you want to turn on StatsD daemon?
Why not? Yes.
So I press one every time.
What port do you want this daemon to listen to?
So we'll just keep the default choice.
Then what is the correct interval?
We'll just keep it as default as well.
So that means that the CloudWatch unified agent
can also collect stuff from StatsD.
Then we have, do we want to have aggregations?
Yes. We'll aggregate them at 60 seconds interval
which is the default.
Do you want to collect metrics from CollectD?
And it must be installed or the agent will fail to start.
So it turns out that I don't have CollectD
on this instance, but it is possible for us
to enable this if we wanted to
but because if we don't have it installed
the agent will fail to start.
We're just going to reply no to this answer.
So I will enter two.
Do you want to monitor any host metrics
such as CPU, memory etcetera?
Yes.
And as you can see, using the CloudWatch unified agent
I am able to have access to memories.
So Ram is a metric you can access
from within the EC2 instance.
We can also collect CPU metrics at the core level.
Again, something this is only possible
by doing metric collection from within the EC2 instance.
Do you want to add EC2 dimensions?
Absolutely. Yes.
And do you want to aggregate them?
Why not?
Then we have the high resolution metrics.
So in case we have that
we have the ability to customize
which resolution will go on them.
So one second, ten seconds, 30 seconds or 60 seconds.
Now to keep cost under control
I will keep it as 60 seconds.
As you can see we can have a resolution
down to one second.
Now, which default metrics config do you want?
Do you want basic, standard, advanced or none?
So we'll keep the default as basic.
And here now I have an entire JSON document
of the configuration we just did.
Which summarizes all the answers we have selected
for the configuration of this CloudWatch unified agent.
So this is good enough. You can have a look.
Are we satisfied with the config?
The answer is yes.
Now, do we have any existing CloudWatch log agents
from configuration file for migration?
The answer is no.
So there used to be before the unified agent
there used to be an old thing called
the Amazon CloudWatch log agents.
And if you had that we could import it
but we don't have this right now.
So we'll say no.
And as you can see
now the unified agents does both metrics
and log at the same time.
Now, do you want to monitor any log files?
We'll say yes.
And now we need to enter the log file path.
So for this is going to be var/log/httpd/access_log
which is the first log file I showed you.
And the group name is going to be access_log for it.
And the log screen name is going to be perfect as well.
And the log retention in days
we could do minus one, for infinity
or we could say, for example, five for seven day retention.
And do you want to additionally specify other log files?
The answer is yes. I want to.
So we enter the path again.
So var/log/httpd/error_log.
Default group name we will keep.
Default stream we keep.
And then again, we'll have a seven day retention
so I'll just enter number five.
Okay. Do you want to specify any more logs we've done?
So just no.
And now we have another config right here.
So now this contains some configuration
for the logs, for our instance.
So this is good enough and I'm good to go.
So do you want to store this config now
in the SSM parameter store?
We will answer yes.
But it will not work
until we have the correct permissions.
So currently this config file has been located and stored
at this address right here, which is great for us
but we want to send this into the SSM parameter store
so that it can be retrieved
by other EC2 instances at run time.
So to do so, we need to store the config.
We need to store the config, this one
in the SSM parameter store
And to do so we need the right permissions.
But if we go into the role we created
the Amazon EC2 role for CloudWatch
which by the way we did not attach to the instance.
Now that I just remembered it.
So let's go back here.
And of course, we'll do security
and then modify IAM role
and we'll need to attach
this Amazon EC2 role for CloudWatch that I just created.
So you see, comes full circle.
So this is perfect.
Even this role,
if we look at the permission itself
and we look at SSM
it's allowed to get a parameter
but not to send a parameter.
So on one instance, we need to add the permissions
to attach a policy
and I will just type CloudWatch agent again
and this time we'll attach the admin policy
which has sufficient permission to put the parameter
into SSM.
And this is only needed when you do the setup
of the CloudWatch unified agents.
So we'll attach this policy. Now we have two policies.
And now my EC2 instance
should be able to perform this next action.
So I will say, yes.
Do you want to store it into the SSM parameter store?
So, yes.
And then what parameter name do you want to use?
So we can just use the default choice.
Amazon Linux, Amazon CloudWatch, minus Linux, press enter.
And then which region do you want to store it in?
So EUS one, which was retrieved directly from
the EC2 metadata by this wizard.
Perfect.
Now which credential should be used?
So the one from the SDK. I press enter.
And then it says successfully put config to parameter store
as Amazon CloudWatch Linux and the program exits now.
So that means that if we go into Amazon Web Services
and then we go into the SSM parameter store.
So let's go into systems manager.
And then find the parameter store.
As we can see now I have one standard parameter
called the Amazon CloudWatch Linux.
And if I click on it
and I look at the value, as you can see
the value is this entire JSON file
that was generated by this wizard.
So it's amazing.
So now the wizard is running and then this has been inserted
into the SSM parameter store.
And now we want to boot up from this parameter.
So back onto our instance, let's clear this screen.
So let's assume this is a whole new instance, okay?
And what we want to do is to just boot up
from this configuration from the SSM parameter store
and start sending metrics and logs into CloudWatch.
So to do so let's go back into our code.
And for this, we have a few options.
So this one can be deleted
because we don't use the CollectD.
So we have options here.
So we can either start directly
from the SSM parameter store.
And this is this command right here.
So the Amazon CloudWatch agent CTL to fetch a config
from SSM at this address right here.
And it's going to fetch the config
we have inserted right here.
So this should work.
The other option is to instead provide a flat file.
So config the JSON file that has been generated
using the wizard, for example
and then the agent can start the quiz
from this static configuration file.
But if you plan to use multiple instances
start from the same file then
it may be a good idea to store it into SSM.
So I will use the command that boots up from SSM.
I will paste it, press enter.
And as you can see, it says
that the configuration version is succeeded
and then it should be good to go.
So let's have a look and see if anything works.
So let's go into the CloudWatch agents.
So let's go into the CloudWatch service.
So CloudWatch.
Perfect.
Let's make sure to use the same region
my EC2 instance is in, which is EU-west-1.
And then I will go into log groups
and underneath we find access log and error log
which are the two logs created by my unified agents.
So under access log, I can find one log stream
which represents my EC2 instance.
And as you can see
we have the two events we saw from before.
Now the cool thing is that if I refresh this page right here
one or two or three times
at some point
the events are going to end up in this log.
So as you can see now
the events have been written by the CloudWatch log agents
unified agents
into this log right here, which is amazing.
And again, if you went into the error log
you will find the log stream right here
that contains the error log stream, if we need to.
So this is perfect.
Now what about metrics?
So if we go into all metrics, we can find a name space
custom name space called CloudWatch agents, CW agents.
And then we can go, for example, by instance ID
and here find the memory used percent
which represents how much ram is being used
by my EC2 instance.
Or the disc used percent and so on.
And you can explore the different metrics.
Here we have the mem used percent again,
but by image ID and with other dimensions and so on.
And here we have six metrics available to us
that are right here.
And it's possible for you to explore the documentation.
And here you can find at some point
you can find all the metrics
that are collected by the CloudWatch agents
they're right here
and the explanations are self explicit.
But you can, just to summarize
collect some CPU metrics, which are very handy.
You can collect as well, some disc space
and IO and so on for your disc.
You can also get information around your mem
so active, available, cached, free and so on.
This is very helpful.
Some network information.
So a number of packets in and out and so on.
And then information about your processes
running on your EC2 instances.
So how many are idle, how many you're running
et cetera, et cetera, which is very, very helpful.
And finally, the swap that you have
on your machine free, used and so on.
So that's it, we've successfully configured
the CloudWatch AI unified agent.
We stored the configuration in SSM and retrieved it
and we're good to go.
Okay. So just to be done, if you are in the DevOps course
you need to keep this instance running.
But if you are in the CSOPs course
then you can terminate this instance
and you'll be good to go.
All right, that's it.
I hope you liked it.
And I will see you in the next lecture.
Can't find what you're looking for?
Get subtitles in any language from opensubtitles.com, and translate them here.