Getting closer, figured the #include is a curl get so I added a file:////tmp/prod-cloud but it's still not reading it in..

Cloud-init v. 0.7.5 running 'modules:config' at Sun, 15 May 2016 02:48:27 +0000. Up 34.14 seconds.
Cloud-init v. 0.7.5 running 'modules:final' at Sun, 15 May 2016 02:48:27 +0000. Up 34.57 seconds.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 959 100 959 0 0 13797 0 --:--:-- --:--:-- --:--:-- 13898
/var/lib/cloud/instance/scripts/part-001: line 11: file:///tmp/prod-cloud: No such file or directory
-rw-r--r--. 1 root root 1830 May 15 02:48 /tmp/prod-cloud
Cloud-init v. 0.7.5 finished at Sun, 15 May 2016 02:48:28 +0000. Datasource DataSourceEc2. Up 35.82 seconds

So the /tmp/prod-cloud is an ls I have in my user-data, so the file is there!! Oh, I guess unless #include is still not firing!! Okay maybe try some multipart trickery here.... (just don't see how that is going to work)

Well that's strange, so tried to multipart it so that it would be able to curl local. But now cloud-init never seems to finish and the host is inaccessible to me.

mutlipart config

Content-Type: multipart/mixed; boundary="===============0432566452=="
MIME-Version: 1.0

--===============0432566452==
Content-Type: text/x-shellscript; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="boot.txt"
#!/bin/sh

1. Grab the curl_s3 file in order to grab the protected cloud config
   

curl -O "http://bucket.s3-us-west-1.amazonaws.com/Public/curl_s3"

1. run the referenced curl_s3 to grab our prod-cloud configuration from a private bucket
   

sh curl_s3 "IAM role" "bucket/Prod/cloud-minimal" "/tmp/prod-cloud"
echo "hello tory" > /tmp/tory

--===============0432566452==
Content-Type: text/x-include-url; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename="include.txt"

file:///tmp/prod-cloud

===============0432566452==

And the host based on system log, does not complete it's cloud-init and thus doesn't create the SSH user/key, so I can't access the node, keeps prompting for password, which we know is not set

system.log

#####################
[ OK 0m Started Initial cloud-init job (pre-networking).
Starting Initial cloud-init job (metadata service crawler)...
http:// 24.582846 cloud-init2056: Cloud-init v. 0.7.5 running 'init' at Sun, 15 May 2016 04:10:38 +0000. Up 24.52 seconds.
http:// 24.693352 cloud-init2056: ci-info: +++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++
http:// 24.697890 cloud-init2056: ci-info: +--------+------+---------------+---------------+-------------------+
http:// 24.702682 cloud-init2056: ci-info: | Device | Up | Address | Mask | Hw-Address |
http:// 24.708158 cloud-init2056: ci-info: +--------+------+---------------+---------------+-------------------+
http:// 24.711779 cloud-init2056: ci-info: | lo: | True | 127.0.0.1 | 255.0.0.0 | . |
http:// 24.719429 cloud-init2056: ci-info: | eth0: | True | 100.99.64.170 | 255.255.255.0 | 06:ed:33:a2:ee:f1 |
http:// 24.723517 cloud-init2056: ci-info: +--------+------+---------------+---------------+-------------------+
http:// 24.727215 cloud-init2056: ci-info: +++++++++++++++++++++++++++++++Route info++++++++++++++++++++++++++++++++
http:// 24.739695 cloud-init2056: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
http:// 24.743364 cloud-init2056: ci-info: | Route | Destination | Gateway | Genmask | Interface | Flags |
http:// 24.747123 cloud-init2056: ci-info: +-------+-------------+-------------+---------------+-----------+-------+
http:// 24.755840 cloud-init2056: ci-info: | 0 | 0.0.0.0 | 100.99.64.1 | 0.0.0.0 | eth0 | UG |
http:// 24.759509 cloud-init2056: ci-info: | 1 | 100.99.64.0 | 0.0.0.0 | 255.255.255.0 | eth0 | U |
http:// 24.763795 cloud-init2056: ci-info: +-------+-------------+-------------+---------------+-----------+-------+

CentOS Linux 7 (Core)
Kernel 3.10.0-327.10.1.el7.x86_64 on an x86_64

ip-100-99-64-170 login:

##################################

So as you can see cloud-init has not finished and based on the last 7 instances and 2 hours of watching, it never will and thus this is busted.

So what is holding it up, my cloud-minimal is as simple as

###
#cloud-config
output: {all: ">> /var/log/cloud-init-output.log"}

packages:
- wget
#####

No reason that won't work!! hahahaha , there is an answer in here some where!!

Tory

I've failed. 0.7.5 cloud-init causes freezes, crashes with some of my attempts (this is fixed in 0.7.7 which CentOS *, does not have.. No idea why it's taking over a year to get 0.7.7 in other distro's.

Anyways, while this was a fun challenge, I didn't get it to function. The #include is not handling the get of a file:/// correctly (and I've not been able to locate the src code that handles the #include so I could see what it was doing with the curl and thus the response.

I finally (which I should have done from the beginning), was to just open up the S3 private bucket from my VPC. So I can just do a #include and direct to my private S3 bucket, that is allowing access based on the VPC..

Note; I initially tried the various examples on the web to allow access from the vpc/vpce, they use a DENY and a notlike statement, that overrides the user/root access, so you end up with no access from the console. And if you decide to just delete it , you lock yourself completely out of your bucket (I did that twice).. So by adding an allow from your vpc, it will accept the inherit user/root credentials.

Whatever, I got it to work, but would have loved to have succeeded with the mutli part Script/include.. to pull in the required auth from my instance IAM file, but it's not working