I wanted to use this new tool to visualize some of the search terms on the topics my group is working on. We are making a tool for producers and storytellers to work through and design their experience in the correct new medium of their choice. This includes Virtual Reality, Augmented Reality, and Mixed Reality.
I started trying to connect some dots on this infographic program (InfraNodus) that scours the internet and creates connections between the nodes and their corresponding concepts/metadata. I tried inputting and searching for different terms. It was interesting to see the connections made by VR, AR, MR and Storytelling.
Some of these connections show the connected nature of these new mediums. For example blending reality with the virtual and the use of computers and their environment. Is this the perfect environment for all new stories? Definitely not, but our tool will help creators pick the perfect medium for the story they are telling.
The preliminary votes/responses from my group’s survey
Rani started this survey and we’ve gotten some prelimary responses already.
These responses are very interesting pertaining to the use of social media and AR. This seems to be the way to gather the largest audience for your work.
A key response here is the ease of use for AR. This is definitely a big advantage as it is definitely more accessible for the end user. The downsides are often the quality/accuracy of the experience itself.
We will be reaching out to experts in the field like the AR/VR association to gain some insights as well.
These are the main stakeholders we are targeting for our research project.
For the last part of my time capsule I decided to first start out sketching some potential scenes. I was having a hard time finding interesting revelations from whoever finds the time capsule. I was almost more interested in trying to predict what the world in 2145 would look like.
I’m not super confident in my drawing skills so I started looking online for comic creators to help me make something that would be enjoyable to read. I tried this site first but found the character editor wasn’t very good and you couldn’t edit their poses.
I was able to find the website Storyboardthat which turned out to be an awesome editor with just enough customization. I began building the scenes and working around some of the objects and backgrounds that they had available. It was honestly really fun to play around with and they give students a 14 day free trial so I could export everything.
I exported the final comic in multiple formats including a gif of the entire thing playing, I attached it at the bottom.
I decided to take this week to work on some of my physical computing skills in designing the enclosure for my time capsule. I happened to receive a mystery box of electronics that I thought would add a really fun element. Also I chose it because this box from Adafruit is something that represents myself and the time period we are in (electronics for hobbyists in 2020). I received a matrix portal in addition to a large 64×32 pixel display. I knew that it would be fun to attach the display to the enclosure. In researching what projects people made with these items, I decided I wanted to put “2020” along with a video of myself in Bitmoji form. Memojis are avatars that people create for use with Apple devices. I’m in a class called performative avatars and we’ve studied how this is another way people choose to represent themselves online. What could be a better way to introduce the viewer to myself than a pixel display I received this week displaying my memoji.
I hit many issues in creating this, but it started with setting up circuit python which was a new platform for me. Thanks to my past experience with Arduino I was able to work through the issues I encountered because I am used to updating libraries and troubleshooting. The thing that was the most confusing was a serial system they use called REPL, this allows you to run singular lines of code while the device is plugged in. In Arduino you usually run the program and can watch the serial monitor but not be able to interact with the device (hence stopping programs I was trying to run).
I first had to wire the matrix portal to the 64×32 pixel display and attach power.
In order to put the bitmoji of my face on the display, I needed to first record myself. I then exported and converted that video to an .mp4. This allowed me to edit it in Adobe Premiere Pro. Once in Adobe Premiere I made the sequence match the dimensions needed for the pixel display (64×32). I used a pixelated font to match the display as well.
It was hard to tell exactly how it would look because it changes at different distances. This was also my first time working with such a small canvas size in video.
I then needed to use a program called Aseprite, to turn my image sequence (I exported a JPG sequence from Premiere) into a vertical bit map. The vertical sprite sheet bitmap is what the circuit python template I was using already worked with. I did some tests to find the correct orientation.
If I had more time I would try and add some audio, along with building a better enclosure to hide the electronics a bit. I would also look into ways to make it more protected.
I was very excited to get started on my time capsule. While it seemed daunting to choose only three or five items, I loved the idea of curating a box for the future. I looked into Andy Warhol’s time capsules and was inspired by breath of items he included. It was a deep look into the mind and process of one of America’s greatest artists.
A theme I have been exploring during my work is the continuity of old and new. I’m fascinated by analog technologies and how they were driven out by digital competitors. I love to see the artifacts of past technologies either ingrained in newer society or fading away and ask why.
My family came to America in 1900 and lived in Newark, New Jersey for quite some time. During this era Newark was basically a little New York, immigrants either chose New York, Newark, or Pittsburg at the time. It had a bustling downtown and city center. I can still see the sparkle in my aunt’s and uncle’s eyes when they talk about the way Newark used to be. Today, Newark has been struggling to revitalize for the latter half of the century. Seeing old photos of Newark always fascinates me. I’ve read census documents, marriage records, and discovered old photos. It’s incredible to try and piece together the past. Today we have tools like google maps where I can revisit old addresses I find to see what the buildings of the past look like now. I know likely many things will be lost in translation for those who don’t have a sentimental attachment to the area. I think a time capsule is a unique opportunity to make a commentary on the past, the things that were important to you in your daily life, and the things you want to pass on. Upon this I feel a responsibility as well to honor the past.
Living in 2020 has been a pretty crazy experience to say the least. Much of our news and content we absorb online, which is another reason I love the idea of using physical items to express our condition.
9/11 Memorial rock
My first item is a rock that I bought from the 9/11 memorial. It states “NO DAY SHALL ERASE YOU FROM THE MEMORY OF TIME.” (Virgil) I really love this rock because it’s in reference to the installation at the 9/11 memorial. I was in NJ at the time while my dad was working in Manhattan. Everyone who was here during that time always remembers that day. It is a day that’s magnitude could not be overstated and is a symbol of rebuilding but honoring the victims. This was a moment New York had to come together and rebuild itself and will always leave its mark on us and our world.
pentax 110 auto camera & Minolta 16 Film
The Pentax 110 auto is my favorite little camera ever made. As I’ve said I’m someone who lives with the idea of the past closer to the forefront of my mind. I think that this camera says a lot of the kind of person I am and could be very interesting for someone to look back in the future. While we’ve had technological advances into digital photography, there is still a large group of people keeping film alive. Maybe in 50 years film will be completely obsolete (I truly hope not). I believe the underlying beauty of a camera like this is there is no getting away from the true art of photography and the photographic process. In digital cameras, abilities to do everything automatically, even sending data directly to a computer over wifi makes them extremely convenient. Sometimes, with so much convenience one loses out on that process. I believe there is a time and a place for both, but I absolutely love to hold onto this process for as long as I can. Also, being a 110 camera means it’s actually an outdated film in 2020. I hold on to this because I believe there is still a lot of inherent value in the smaller format. While attempting to differentiate myself as a photographer I think we have to look backwards as well.
The Minolta 16 film is another item that goes along with the Pentax 110 Auto. It is the same smaller format (110/16mm) and it was custom cut from a gentleman on Ebay. He even made his own packaging. I found this fascinating that someone would take the time to try and keep this old film alive. I surely treasure that there are people out there doing so.
Face mask with face scanned part
I recently used an application called Bellus3D to scan my face using photos, true depth data from the iPhone, and their wrapping algorithm. This resulted in a pretty accurate 3D model of my face. I chose a mask because obviously it’s very important in our present day Covid world. I chose to include it with the face scanned part to highlight a movement of people using modern technologies to help. I saw an extreme push from people all over the world flock to the maker community to find ways to help. This included sewing masks, people making 3d printed masks with air conditioning filters etc. This was extremely heartwarming and inspiring and it’s definitely something I want to be remembered in the time capsule. When we saw our doctors and nurses suffer from a PPE shortage, many jumped in to help in any way they could using their skills and what was available. I believe as 3D printing technologies get better and the consumer is able to print more homemade parts we will see this self sufficient part of society grow.
Proven skincare pamphlet and bottle
I think the products of 2020 in America are very specific and I wanted to find something that I thought embodied what that means. I chose Proven Skincare because I recently saw them on Shark Tank, another phenomenon that I think will be interesting to look back on. They came out with the idea of using an algorithm to create a perfect skincare product based on a number of key factors. I feel this represents capitalism in America, but also the idea that we trust machines/algorithms more these days to create personalized items. Data privacy and tracking are huge concerns these days, and carry both good and bad outcomes. We have created artificial intelligence to create profiles of our interests. I wonder how this will continue in the future, and I think this product is a really telling example of where capitalism and technology in America are right now.
The photo essay assignment required that I pick one of their prompts in addition to the object I used in my 50 renderings assignment (a camera). The protagonist that stood out to me was the colorful butterfly. The reason I chose this was because I’ve been wanting to shoot some black and white photos recently and I thought it could be an interesting collage of color, to use an animated butterfly along with black and white photography.
A butterfly protagonist with a camera was definitely a challenge to come up with a story for. So I started with what I had, a New York City backdrop with black and white photography.
It was just like any other day for National Geographic Photographer Bobby the butterfly. He was on assignment in New York City, tasked with capturing the city skylines. He had always been interested in landscape photography his whole life. The scenery captivated him, and working in black and white was his preferred method of shooting. He swears by Ansel Adam’s book “The Negative.”
Setting up his first shot he couldn’t help but be bored with cloudless sky. There were no gradients in the mountains, only greenery and building reflections. He was uninspired to say the least. Maybe for the first time in his career, he was doubting the format in which he grew up using.
While setting up a shot using the railing on a balcony he noticed a fellow butterfly, one with colors he had never seen before. For the first time in his life, Bobby was seeing color. “I can’t believe the most beautiful butterfly I’ve ever seen, is in New York!” he thought to himself. Bobby needed to see if he could ask for a photo, it was not his usual style but for the first time he wanted to shoot in color.
Bobby switched lenses and chased after her, almost flying into foliage on the way! He was worried he had scared her off, after all even butterflies do not like the paparazzi.
Bobby lost sight of her and diligently went back to work. All day shooting landscapes, but he couldn’t get the image of the butterfly that got away. He even stopped to take a picture of a dilapidated plant. This is how Bobby was feeling inside, out of his comfort zone and decaying into his own sadness.
After a long day of work bobby decided to rest on a chair outside and drink a nectar-beer. He had finished his work for the day but still couldn’t get the other butterfly out of his mind. To his surprise, she approached him out of nowhere stating she was camera shy and didn’t appreciate him chasing after her with his camera! Bobby learned a valuable lesson that day, sometimes the best moments are experienced with his camera in his bag.
For this week’s assignment I decided to use the new model exercise to imagine new ways to think about capture devices. While my thesis will be largely centered around analog photography, I want to also explore new ways current technologies could take capturing forward.
What exists now?
In the landscape of virtual production, we have many new tools for creation. These tools however are often driven by large companies who follow large leading productions/studios. An example of this being Star Wars/Avatar/Blockbuster films that can invent new strategies for filmmaking/capture. Live production with LED walls has become a huge market recently as well. Some of the technologies they employ are camera data/camera location/tracking backgrounds/live updating backgrounds. Also Apple has introduced the true sense tracking and depth information for apps to use. Traditionally, photography cameras only have technology related to exposure control, automatic features, film advancing, and now WIFI enabled sharing. Would depth data bring any new features to these photos? What possibilities could be found with integrated different kinds of capture technology and data?
To explore this I used a card deck from Dan Lockton’s New Metaphors.
The cards that stuck with me:
The presence of AI & The opportune moment
Privacy of your data
The backstory of a product or service
In the concept of a camera/capturing device how can these concepts and metaphors be utilized? I did some sketching to think through these concepts.
For adaptability I really was interested in the idea of a modular format camera. This currently is impossible but was a fun thought experiment. The biggest barriers are the fact that flange focal distance for different formats can change dramatically. I think the use of bellows and interchanging film backs could be one solution. I was inspired by the Hasselblad modular design that allows film backs to be changed, for different types of film (not format/size).
I think there is something DIY and magical about the use of bellows in photography. I think it could help with the creation of newer low budget medium format cameras, as this was the case in the past. Collapsible for convenience.
I was able to use my phone and have my roommate scan myself for this week. I tried all of the different capture apps for Iphone and found one used for Dentists called Bellus3D provided the best mesh/high resolution scan of my face. It was a little more expensive to export the scan but their algorithm for wrapping the head and mesh was far superior to the others. There was no cleanup and I found the other apps would result in messy meshes with more sparse point clouds. This system actually uses a variety of poses as opposed to working around in realtime while trying to focus on keeping the mesh clean. It then runs its system based on the photos and outputs a very clean model.
This week we have to translate digital to physical affordances, and I chose the Twitch Online platform. I wanted to explore the physicality of these interactions but also explore how this platform could add more interfaces as well.
I want to look at interactions on both sides of this platform, creators & audience. Moving forward I think many types of live entertainment will become more interactive and much of that is built upon twitch.
The main areas of interaction are in this right hand side of the frame. There are a few ways to do so.
A very literal scroll. My favorite part about twitch is the way broadcasters interact with the infinite scroll of comments. This is especially satisfying because it is displayed for everyone including the broadcaster. You know that the broadcaster can see it and I think it would be great to have it exist in physical space to solidify that. I printed out some comments to demonstrate the scroll after users type in to the stream.
Emojis are a simple yet effective way of more passive viewership. Without physical presence this is a great way to get feedback.
The happy or not stand that many see at airport terminals and DMVs would be a perfect physical way of responding with emojis. Each twitch platform can actually create custom emojis as well to express more.
Navigating cameras and the screen
The way twitch works now is that broadcasters will change their cameras based upon where they want the viewer to look. I think this is an interesting opportunity to add more physical interaction. The viewer should be able to toggle and change the position of the camera (digital movement of overall view) based upon their own physical switches and joystick.
I think overall it would be a really interesting experience to be able to physicalize a lot of the affordances on this online platform and if a lot of development was made there could be a whole host of new interactions.
For the short film assignment I wanted to play around with version 4.26 of Unreal Engine. I was very excited to find a brand new environment available for free on the marketplace that I could use as my setting for experimenting. I also wanted to bring back the old man with the cigar character.
When I tried to load in the pre built worlds I was running into space issues even though everything was on my separate drive, some things were constrained to my local disk. This along with the large textures kept crashing the site. Instead I began working with an HDR background texture which looked realistic and was fun to play with and I began working in the sequencer to create some shots. I was definitely a little in over my head, trying to animate the cine cameras and I’ve realized 24 fps looks quite bad coming out of unreal. I need to keep working on creating animations and spawning events.