At startup first log directory
/u01/app/oracle/gc_inst-r5/user_projects/domains/GCDomain/servers/EMGC_OMS1/logs/EMGC_OMS1.log
After that log directory
/u01/app/oracle/gc_inst/em/EMGC_OMS1/sysman/log/
Note: By setting memory to low, like below OMS will hang and never complete startup

emctl set property -name  'JAVA_EM_MEM_ARGS' -value '-Xms256m -Xmx1740m'

How to increase the memory on EM 12c
https://community.oracle.com/community/support/support-blogs/enterprise-manager-support-blog/blog/2015/01/20/how-to-tune-the-jvm-heap-memory-arguments-for-12c-oms-and-agent