Morning! welcome to virtualcloudblog.com and thanks for checking it out. Today, I’ll write this post about Arista and why /var/log/lastlog is HUGE, but firstly let me introduce Arista
About Arista
Arista Networks was founded to pioneer and deliver software-driven cloud networking solutions for large data center storage and computing environments. Arista’s award-winning platforms, ranging in Ethernet speeds from 10 to 100 gigabits per second, redefine scalability, agility and resilience. Arista has shipped more than 15 million cloud networking ports worldwide with CloudVision and EOS, an advanced network operating system. Committed to open standards, Arista is a founding member of the 25/50GbE consortium.
Arista Networks products are available worldwide directly and through partners and is a leader in building scalable, high-performance and ultra-low latency cloud networks with low power consumption and a small footprint for modern data center and cloud computing environments. It has been awared by Gartner as a Leader in its 2017 Magic Quadrant for Data Center Networking.
Why /var/log/lastlog is HUGE
After login in the Arista switch, the system shows a warning.
|
1 2 3 4 |
Last login: Tue Jun 12 13:04:09 2018 from 10.10.172.3 Warning: the following filesystems have less than 10% free space left: tmpfs (on /var/log) 7% (28M Available) Please remove configuration such as tracing and clean up the space. |
so it seems related to /var/log
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
labswitch03#bash Arista Networks EOS shell [admin@labswitch03~]$ df -h Filesystem Size Used Avail Use% Mounted on none 557M 43M 514M 8% / none 557M 43M 514M 8% /.overlay devtmpfs 8.0M 0 8.0M 0% /dev tmpfs 1.9G 0 1.9G 0% /dev/shm tmpfs 1.9G 1016K 1.9G 1% /run tmpfs 1.9G 0 1.9G 0% /sys/fs/cgroup tmpfs 557M 8.0K 557M 1% /tmp tmpfs 64M 708K 64M 2% /.deltas tmpfs 1.9G 0 1.9G 0% /var/run/netns tmpfs 371M 0 371M 0% /var/core tmpfs 371M 346M 26M 94% /var/log tmpfs 1.0G 17M 1008M 2% /var/shmem /dev/mmcblk0p1 3.7G 1.7G 2.0G 47% /mnt/flash |
as we can see /var/log is 94% used, which is not good. But the lastlog is “using” 30 GB!!!!!
|
1 2 |
[admin@labswitch03]$ ls -ltahr /var/log/lastlog -rw-r--r-- 1 root root 30G Jun 13 06:41 /var/log/lastlog |
This is because space is “allocated” ahead of time for all possible user IDs, which is about 232 users multiplied by 256 bytes for each login record. This is, thankfully, an illusion. The lastlog file is created as a sparse file, so only the chunks of the file that are used actually take up physical storage space. So all the space really isn’t allocated.
To see how big the file really is use the “du” command. In this case, the log is 72K
|
1 2 |
[admin@labswitch03]$ du -h /var/log/lastlog 72K /var/log/lastlog |
Workaround
It’s related to the a bug which is hitting the BUG-177893 introduced in 4.14.5 and fixed in 4.20.5 OS.
Release Note: “On every Event Monitor sync or show command, eventMon database at /var/log/eventMon.db will grow without bounds until user explicitly clears the db using event-monitor clear command.”
- To retrieve the space you will need to clear the current entries from the database tables using the following command.
1(config)#event-monitor clear - Once this is done, we should be able to see the file size decrease for the eventMon.db file in /var/log directory.
- We can limit the number of records to store by adjusting the default backlog for event-monitor.
1(config)#event-monitor backup max-size 100
BTW: The clear command is not impacting. It is just clearing the logs and it can be run at anytime
Thanks for sharing!
Ryan
Sergey