Why aren't my IIS Logs being copied for my Azure web role?

Question

This is a follow up on this question. I used Cerebrata Diagnostics Manager Remote Diagnostics to try to turn on IIS logs. I hadn't deployed with it on. It seemed to work and a few files were copied. Then it never seemed to work again. I tweaked settings again. I tried deleting the iis related blob and table storage entries to see if that would get it to start over. Here is what the config looks like in the wad-control-container that shows that it seemed to be updated based on Cerebrata tool.

<?xml version="1.0"?>
<ConfigRequest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <OnDemandTransfers />
  <DataSources>
    <OverallQuotaInMB>4096</OverallQuotaInMB>
    <Logs>
      <BufferQuotaInMB>1024</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>1</ScheduledTransferPeriodInMinutes>
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </Logs>
    <DiagnosticInfrastructureLogs>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </DiagnosticInfrastructureLogs>
    <PerformanceCounters>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <Subscriptions />
    </PerformanceCounters>
    <WindowsEventLog>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>0</ScheduledTransferPeriodInMinutes>
      <Subscriptions />
      <ScheduledTransferLogLevelFilter>Undefined</ScheduledTransferLogLevelFilter>
    </WindowsEventLog>
    <Directories>
      <BufferQuotaInMB>0</BufferQuotaInMB>
      <ScheduledTransferPeriodInMinutes>1</ScheduledTransferPeriodInMinutes>
      <Subscriptions>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\8973cd09642f4dfeafe830612cc8c1fe.AllRole.DiagnosticStore\FailedReqLogFiles</Path>
          <Container>wad-iis-failedreqlogfiles</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\8973cd09642f4dfeafe830612cc8c1fe.AllRole.DiagnosticStore\LogFiles</Path>
          <Container>wad-iis-logfiles</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
        <DirectoryConfiguration>
          <Path>C:\Resources\directory\8973cd09642f4dfeafe830612cc8c1fe.AllRole.DiagnosticStore\CrashDumps</Path>
          <Container>wad-crash-dumps</Container>
          <DirectoryQuotaInMB>1024</DirectoryQuotaInMB>
        </DirectoryConfiguration>
      </Subscriptions>
    </Directories>
  </DataSources>
  <IsDefault>false</IsDefault>
</ConfigRequest>

Any ideas on why it won't seem to work?

UPDATE

We redeployed today with the following diagnostics.wadcfg and still no IISLogs. Trace logs are working. We don't have any code that calls the diagnostics because it was my understanding that the file could handle it all. Am I missing something?

<DiagnosticMonitorConfiguration xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration"
      configurationChangePollInterval="PT1M"
      overallQuotaInMB="4096">
  <Logs bufferQuotaInMB="1024"
     scheduledTransferLogLevelFilter="Verbose"
     scheduledTransferPeriod="PT1M" />
  <Directories bufferQuotaInMB="1024"
   scheduledTransferPeriod="PT1M">

    <!-- These three elements specify the special directories 
           that are set up for the log types -->
    <CrashDumps container="wad-crash-dumps" directoryQuotaInMB="256" />
    <FailedRequestLogs container="wad-frq" directoryQuotaInMB="256" />
    <IISLogs container="wad-iis" directoryQuotaInMB="256" />

  </Directories>
</DiagnosticMonitorConfiguration>

Could it be that the web roles are XS instances (since we're just testing right now)? Again, it did work once, but seems to be dead now.

Answer 1

Few suggestions:

1. Lower the OverallQuotaInMB to something like 4000

2. Increase BufferQuotaInMB under the Directories node to a number (say 1 gig)

3. Lower the other individual Directory quotas so that they add up to something slightly less than BufferQuotaInMB in #2 and so that ALL of the quotas (including the overall Directories and and individual Folders are under OverallQuotaInMB). IE: Logs - 1gig (this is trace data), Directories: 1 gig, FailedRequests: 256mb, IISLogs: 256mb, CrashDumps: 256mb

4. Reboot your servers (just in case)

5. Good luck

Basically, I've seen diagnostics act fussy when the overall quota is set to the max space that Azure allocates for diagnostics storage (4gig). Lowering individual quotas so that they add up to something less than total quota also helps because if Azure Diagnostics ever fills up, there is breathing room before Azure removes old data.

Overall, setting up Azure Diagnostics is something of a black magic art. I've been helping AzureWatch customers do this for two years now and I still feel like I'm fumbling with the quotas. Wish they would just let users turn the thing on or off have the entire config be driven by convention vs. configuration. Almost noone ever cares to capture the data onto their VM's and not transfer it to azure storage and thus small quotas are totally fine for majority of the cases as majority of folks transfer their data to storage every few minutes.

HTH