kubectl logs to any pod, you’re querying the
stdout/stderr streams sent to the node’s log files (
/var/log/*), which in turn depends on the underlying host OS logrotate.
You can try to determine if this is the problem by either, comparing the logs from
kubectl logs versus the ones sent into a logging backend (if any, for instance Fluentd or the ELK stack) or directly SSHing into the node and locating them directly in the logging path.
There aren’t many details on how are you running your Kubernetes cluster, but the information provided strongly suggests that this is a node level issue, specific to how pod logs are being managed internally.
Finally, the approach using
kubectl exec <podname> -- /bin/bash is a good idea to determine if the container is locally storing these logs and the issue is when they’re sent into the host node.
You can use
kubectl logs --follow [Pod name] -c [Container name], this will show you the realtime logs
--follow will let you view logs in real time
Read more here: Source link