(This is optional extra credit that we’ll just tack onto your cumulative
score for the other assignments; this assignment is worth 1/2 of an
assignment)

Write about 500 words as a reaction to one or more of the talks by Zach
Gillen (Kaiser), Ryan Greenberg (Twitter), Thejo Kote (Automatic), Bryan
Rea (Google), or Igor Pesenson (Salesforce). Analyze how their talk
applies to one or more of the 202 lectures this semester and demonstrate
with your cogent use of 202 concepts and your knowledge of the course
readings that you’ve benefited from coming to class.

(This is due before you take your final exam next week, because I want
that exam to be the end of the semester for you. Send your assignment
to me as an email attachment).

(I don’t expect most of you to do this. Numerically this isn’t very
significant with respect to your final grade, and you need to get your
Assignment 10 case study written, and you need to study for the final
exam. But I’m offering this optional assignment as another chance to
think about what you learned in 202, and perhaps those of who didn’t
show me what you knew on the midterm can use this to demonstrate a
little bit more that you get 202…)

Due: Thursday, Dec 12th 2013, 9am

For this assignment, you will write a case study of an organizing system in the domain you have explored throughout this class.  The assignment is intended to help you review and synthesize what you have learned this semester. Treat this assignment as part of your studying for the final exam.

You must select a particular organizing system, not a class of organizing systems, for this assignment. For example, “my personal photo database” would be an appropriate organizing system for this assignment, but “personal photo management” in general would not.

Your case study should have the same sections as the case studies in Chapter 10 of The Discipline of Organizing:

• Overview (1 pt)
• What resources are being used? (2 pts)
• Why are the resources organized? (2 pts)
• How much are the resources organized? (2 pts)
• When are the resources organized? (1 pt)
• Who does the organizing? (1 pt)
• Other considerations (1 pt)

In each section, consider explaining the alternatives and their ramifications. When making each design decision, your system’s designer chose between several alternatives. Describe each of those alternatives, their ramifications and the reasons for the designer’s final choice.

Also, when appropriate, identify the challenges unique to your system. Describe the puzzles, problems or dilemmas your system’s designer faced that are unique to your system. When possible, also describe their solution.

Whenever possible, use the vocabulary of The Discipline of Organizing and this class. For example, instead of saying “We struggled with how specifically to describe each item,” say “We struggled with resource description granularity.” Although you might use a more domain-specific vocabulary in your work, use TDO vocabulary in this assignment to show us you understand it.

Refer to both chapter 10 and the organizing systems design decisions described in chapter one for example case studies and other issues to consider.

Your entire assignment, including section headings, must be 1,000 words or less. The best case studies will be included in future editions of The Discipline of Organizing.

Submission Instructions
Post your case study to the 202 blog at http://bit.ly/1iw9z5o.

Check the “case studies” category on the right and tag your post with your TA’s first name.

(To do this, you will need to log in with your I School username and password. If you have log-in issues, contact Fred at fchasen@berkeley.edu with your I School username. If you run out of troublehooting time, email the assignment to your TA.)

Due Date: Thursday, December 5th, 9am

Assignment Overview

In this assignment, you will:

1. Learn how to use a toolkit for text analysis
2. Learn about stemming and using stop words
3. Create an index for documents in a collection
4. Process a search query and return relevant documents
5. Reflect on your experiences

Download as: Word | PDF

Text files for the assignment:

• A9_text
• A9_text2
• A9_text3
• A9_text4

Submission Requirements

You will submit a file called YourNameA9report.pdf, which will include

• Short answers to each of the reflection prompts below.
• The chart that you will create in Part 4.
• Your vector plot of your search query from Reflection 4.

Instructions

Peter Holme’s word stemmer is a web-based text analysis tool which walks you through the processes of stemming and analyzing documents. These are the steps one would go through while creating an index of a document which can be used for IR.

Part 0

1. Go to http://holme.se/stem/
2. Open A9_text.txt This file contains text you know well, the first three paragraphs of TDO.
3. Read the text, copy it and paste it into the Text field in the word stemmer.

Part 1

• Click the “Don’t use any stopwords” checkbox in Extra super fun stuff.
• Increase the number of words for word count to 100.
• Leave all other settings at their default.
• Click Send.
• Look at the results in Step 1 on the right.

➪ Reflection 1 (1 point)

• Why did some words disappear?
• Why do we want to remove these terms/words and what are some problems that may arise when we do so?

Part 2.1

• Change the minimum number of characters in a word to 1.
• Run the analysis again. Look at the results in Step 1, and note what got added back and what is still missing. Now, look at Step 2 and check out how words were stemmed.

➪ Reflection 2.1 (1 point)

Pick three stemmed words from the list, and for each, answer the following:

• Why was that stem chosen?
• What problems might arise with that stem?

Part 2.2

• Next, look at the Word Count list in Step 4: Notice the number of words and the frequency.
• Are there any words that surprise you in this list?

➪ Reflection 2.2 (1 point)

• How much about the article can be inferred from the words on this list?
• Looking at the word frequency, where would you draw the line to eliminate stop words?

Part 3

• Uncheck the don’t use any stopwords checkbox.
• Take the top 10 words of the resulting word frequency list from the last analysis that you think should be used as stopwords. Enter them into the stopwords box and then run the analysis.
• Copy the resulting Word Count list (Step 4) into a table or spreadsheet.
• Next, delete the list you just added to the stopwords and run the analysis again. (When the stopwords box is empty, it will run using a default list of stopwords, which you can see in Step 3).
• Copy these new Word Count results (Step 4) into your table and compare them with your previous results to see how the list and frequencies changed.

➪ Reflection 3 (1 point)

• Why do you think the system chose some of the stopwords? Are there any words that surprise you?
• Why are the stopwords stemmed?

Part 4

• Run A9_text2.txt, A9_text3.txt, and A9_text4.txt through the system and copy each resulting word count list into separate parts of a spreadsheet. Now you have an index for each document in your collection (note that in a real system you would want the full index – for simplicity’s sake, we’ve cut it off at 100 terms).
• Now you get to play the part of a search engine. You will now manually run the query “Organizing System” on your collection of three documents.
• For each document, find the term frequency (tf) for each term in the query. Record each calculation in a chart like the one below.
• Calculate the idf for each term. List it in your chart and show your work in Reflection 4.
• Calculate the tf-idf for each document. Again, show your work in Reflection 4.

➪Reflection 4 (6 points)

• As indicated before, show us your work in calculating tf, idf, and tf-idf. (2 points)
• In a simple graph, draw the vector plot for your search query (you can do this by hand or in software). (2 points)
• Which document appears to best fit your query? Why? (1 points)
• What does tf-idf allow you to deduce that using term frequency alone does not? (1 point)

Extra Credit (1 point):

• Calculate the cosine similarity between your top two documents and the query. Show your work. Which is now the best fit for your query? Why?

Due: Thursday, 21 Nov 2013, 9am

For this assignment you will analyze the search interfaces of Wine.com and Zillow.com through the lens of organizing systems presented thus far in class. Familiarize yourself with their search functionality by performing a few searches (say “pinot noir” for Wine.com and “Berkeley, CA” for Zillow.com). Screenshots of each website are attached so you can see the general area you ought to be looking at.

Write 450-550 words for each of the following questions (for a total of 900-1100 words). Submit a single file FirstnameLastnameA8Report.pdf that contains all parts of the assignment.

wine.com advanced search: http://www.wine.com/v6/search/advancedsearch.aspx

zillow.com search: http://www.zillow.com/homes/berkeley,-ca_rb/
click on the filter buttons to the right of the search bar to bring up the advanced search boxes, if they don’t show up automatically

Part 1 (4 points)

Analyze the search interface and the search results using the key concepts of organizing systems we’ve covered this semester. DO NOT concern yourself with whether the results are actually good or if they fulfill the user’s information needs. Focus on what these interfaces reveal to you about the underlying organizing system. Specifically, comment on each of the following:

1. Classification principles

2. Resource description

3. Granularity

4. One issue of your choosing.

Provide 1-2 specific examples for each, to illustrate your points.

Part 2 (6 points)

Discuss 3 specific design issues from the Hearst chapter and for each issue, evaluate how well it is satisfied in both applications.

Bonus (1 point)

Suggest a way that either of these search interfaces could be improved, justifying your suggestion using one of the design issues from the Hearst chapter (maximum of 50 words extra). If appropriate, you may use one of the issues you brought up in Part 2.

Screenshots

Due: Thursday, 31 Oct 2013, 9am

Submission Requirements:

• One ZIP file named FirstnameLastnameA7.zip, containing two files:
• One PDF named FirstnameLastnameA7Chart.pdf containing a spreadsheet of parts 1, 2 and 3
• One PDF named FirstnameLastnameA7Report.pdf containing your domain sentence (part 0) and reflection (part 4)

In this assignment you will:

1. Restate your domain and clarify its scope
2. Make a list of potential interactions in your A1 organizing system
3. Analyze the designed interactions and create equivalence classes
4. Further classify them based on a few sets of criteria
5. Reflect on the assignment

INTERACTIONS include any activity, function, or service supported by or enabled with respect to the resources in a collection or with respect the collection as a whole.

Your initial task in this assignment is to identify the potential interactions in your Assignment 1 Organizing System. You will then analyze and classify the interactions, and in doing so you will be refining the scope and scale of your organizing system.

Part 0. Write one sentence telling us your domain and its scope.

We know you already submitted a domain in A1, but you man have changed or refined it, or adjusted the scope. This will give us an idea of where you are with that now.

Part 1. Make a list of the potential interactions in your Assignment 1 organizing system. (3 points)

Be creative here. Don’t be limited to the interactions that you can see in other instances of organizing systems in your chosen domain. Aim to have no more than 10 interactions.

If your organizing system is in a domain with physical resources, you should focus on what TDO calls the “designed interactions” that are enabled by intentional acts of resource description and organization. (The same is true if your domain is digital, but since everything must be designed in a digital domain, it’s less of a potential problem.) We don’t want you analyzing interactions that are completely intrinsic and “non-designed”. (Recall the sidebar example in Chapter 1 about the “Digital Zoo” – if you visit an actual zoo, “viewing the animals” is an interaction that doesn’t take much design effort because “viewing” is pretty intrinsic for physical resources. In the digital zoo, however, instead of “access just by looking” you’d need to design and implement some “access via technology” mechanisms for locating and viewing the remote animals).

Some interactions are quite generic and are probably or potentially supported by every organizing system. Examples of generic interactions could be selecting a resource from a set of candidates in a collection, finding a resource that you already know exists, and discovering the identity of a particular resource. Include these in your list, but contextualize them to make them descriptive and relevant.

Again, keep your list to no more than 10 interactions. If you find yourself coming up with significantly more than that, you may need to adjust the granularity of the interactions you’re choosing, or modify the scope of your organizing system.

Part 2. Analyze the designed interactions and create some equivalence classes for those that are similar in function and or implementation. (2 points)

In part one, you will have come up with a collection of all the interesting interactions you can contemplate for your organizing system. In this part, you will classify them. You will want to pay close attention to the granularity of the interactions that you are classifying. For example, some organizing systems may themselves collect information about every single interaction with its resources, the context of that interaction, etc. This “information collection” interaction may be something you want to record and classify, but you would probably not want to list every possible variation or report each one as a separate interaction.

Record each of your designed interactions and their equivalence classes in a spreadsheet based on the attached sample.

Part 3. Classify the interactions based on the following criteria: (3 points)

Record these in the same spreadsheet as in part 2, in the 3rd, 4th, and 5th columns as indicated by the sample.

A. Classify the interactions into those that are 1) based on specific resources (one at a time) or 2) interactions that utilize collection-level properties (i.e. the collection as a whole). (1 point)

B. Separately (independently from part A), classify the interactions into those that are 1) initiated by a user of the organizing system, 2) initiated by the resources themselves, or 3) performed with “mixed initiative” in which both the user and the resource initiate some aspect of their joint interaction. Explain why you chose each classification when you feel it isn’t self-explanatory. (1 point)

C. Take the interactions which you classified in part A as being based on specific resources. Further classify them into those that involve 1) interacting with the original resource, 2) interacting with descriptions of the original resource, or 3) interacting with copies of the original resource. (1 point)

Part 4. Reflect on the assignment (2 points)
Write a couple of paragraphs reflecting on the assignment, including but not limited to:

• Did the number/type of interactions you found, or the difficulty/ease with which you thought of them, tell you anything about the scope of your organizing system?
• Did you find yourself adjusting your scope in mid-exercise? If so, how did you go about this? If not, moving forward, how might you adjust your scope to make it more manageable in terms of identifying and classifying interactions (if needed)?
• Did you find describing or classifying any of the interactions particularly difficult? If so, what was it about those interactions that made it hard?

Starter spreadsheet for parts 1, 2, and 3: Assignment 7 – Spreadsheet

Due: October 24, 2013, 9 a.m.

The goal of this assignment is to give you practice thinking about categorization and category membership, abstraction, classification, and ontologies. Pretend you are a prehistoric human, just starting to think about abstract concepts. You come across 15 animals, and decide to organize them into a hierarchical classification scheme, using properties and behaviors you can directly observe. (We are setting the stage this way to emphasize that we want you to create an intuitive classification and not a scientific one. We do not want to you consult authoritative biological classifications. Please don’t fret about the impossibility that a prehistoric human might encounter all 15 of these animals).

In this assignment, you will:

• Define equivalence classes for 15 animal instances
• Sort those classes into a hierarchy of animal types
• Create a diagram of your ontology
• Write definitions for each part of your ontology using hypernyms and hyponyms
• Reflect on your experience.

Submission Requirements
You will submit a zip file named FirstnameLastnameA6.zip that includes 3 files:

1. A spreadsheet (modeled on the sample spreadsheet attached here) called FirstnameLastnameA6.pdf
2. Your diagram, which will be called FirstnameLastnameA6.pdf
3. A brief reflection (FirstnameLastnameA6Reflection, in pdf format)

Regardless of what method you use to make your diagram, please submit all of your files as a pdf so we can be sure we’re seeing what you intended for us to see.

Part 1: Identify Equivalence Classes (3 points)
Remember that an equivalence class, also known as category or type, is a way of specifying that a group of resources should be considered the same thing in a given context. With that in mind, look at the 15 animals here: https://www.dropbox.com/sh/bdopy1ntiuzgjiy/6Km8_T8Sah
Using this attached spreadsheet, list the 15 animals using the animal name listed on each file (i.e., Fish, Penguin, Eagle, …).

For each animal instance, identify a possible equivalence class to which the instance belongs. For example, if we were classifying musical instruments, and you were creating an equivalence class for a drum set, you might pick something like “rhythm instrument”. Depending on the other musical instruments you were organizing, you might want a more abstract (instruments) or less abstract (four piece drum sets) equivalence class. You are making a choice about the level of abstraction you use.

As you are making your first pass through the instances, do not worry too much about naming these types. Organizing is an iterative process. You are likely to go back to them and revise them as you progress through the assignment. If the ambiguity is making you a little crazy, our advice is to come up with a placeholder and move on; new naming ideas might surface as you start to arrange your hierarchy.

Focus on what is observable from the image, and not from further research. You may use what you already know about the behavior of the different animals as long as it is something that can be observed (e.g. birds fly, fish swim, giraffes walk).

Part 2. Organize Your Equivalence Classes into a Hierarchy (2 points)
Now that you have identified equivalence classes for each of the animal instances, begin arranging the classes into a hierarchy. The root element of your hierarchy will be “Animals.” The leaf level elements of your hierarchy will be the specific animal instances. When you created your equivalence classes in Part 1, you added a second level to the hierarchy — more abstract than your instances but less abstract than “Animals.” Take this one step further by adding one more level of abstraction — a new level between your equivalence classes and “Animal.” These are hypernyms or “super-types.”

At this phase of the assignment, it’s important that you strive for a consistent level of abstraction among your “super-types” (hypernyms). For example, if “musical instruments” was our root and the next level down included the groupings “clarinets” and “stringed instruments” that might be a sign that our classification system did not maintain a consistent level of abstraction.

Part 3: Create a Diagram or Visualization of Your Hierarchy (1 point)
The visualization does not have to be fancy. We repeat: it does not have to be fancy. Start with “Animal” as the root of your hierarchy, then add your “super-types,” then add your types, and then your animal instances as the final level in your diagram. You can use any tool you wish (including drawing by hand and scanning your drawing), as long as it allows you to represent the hierarchical relations in your ontology.

Part 4: Define your Types and “Super-Types” (2 points)
Now, you will write definitions for both your equivalence classes and your ‘super types’ so an ordinary person would be able to categorize new instances. You’ll be following this formula for definitions:

Hyponym = {adjective+} hypernym {distinguishing clause}

Example:
Clarinets = {reeded} Woodwinds {that are approximately cylindrical in shape and have numerous keys}
Woodwinds = {reed or flute} Instruments {that produce sound when air is blown into them}

Record each definition in your spreadsheet.

Part 5: Reflect on Your Experience (2 points)
In your reflection document, write a few paragraphs about the approaches you used to identify equivalence classes and organize them into “super-types.” Consider your thought process throughout the assignment, tips from readings or lectures you drew on, finding consistency in your layers of abstraction.

Download: Assignment 6 Starter Spreadsheet

Due: October 15, 2013, 9 a.m.

What to hand in

Submit a single file YourNameA5Report.pdf that contains all parts of the assignment. Include your crosswalk, your pseudo-code, and your reflection.

Download: A5-Assignment | A5 Instances 

Due 10/1

This purpose of this assignment is to get you familiar with one of the two editors and understand how an XML instance, schema, and transform fit together.

XML Spy (Windows) – https://www.ischool.berkeley.edu/intranet/computing/software/xmlspy

XML Spy is an award-winning XML editor and development environment that has been generously provided to the I School for use in courses and projects, but it runs only on Windows or on Mac/Linux using a Windows emulator.

oXygen (Mac / Linux) – https://www.ischool.berkeley.edu/intranet/computing/software/oxygen

oXygen XM  runs within a Java Virtual Machine, so it runs on any platform with a Java Runtime Environment. You can use the I School’s license to install it on your own computer.

Part 0:

This section is not graded, but will instruct you on how to use XML and the XML editors. The listed questions don’t need to be answered, but are here to help you think about and understand XML

1. Install XML Spy or oXygen

2. Download the attached zip file, which contains:
   Report.xml: an XML instance
   Report.dtd: an XML Document Type Definition
   Report.xsl: an XML transformation file

3. Open Report.xml in the editor.

4. Also open Report.xml in a web browser. Why is it rendered this way? (“View > Source” on menu bar).

5. Back in the editor, check the XML instance for “well-formedness” – conformance to the syntax rules for XML (F7 in XML Spy; in oXygen, click the blue-checked document icon in the tool bar).
   

6. Delete the beginning <Name> tag. Is the instance still well-formed?
   Change <Para> to <para>. Is the instance well-formed?
   XML is enforcing more restrictive syntax rules than HTML. Or put another way, XML doesn’t allow bad practices that browsers typically forgive with HTML.
   Undo these changes so that your instance is well-formed again.

7. Specify an XML Document Type Definition for the XML instance by inserting

   directly below the

   declaration at the top of the file.

8. Validate the XML instance. (F8 in XML Spy; in oXygen, use the red-checked document icon, near the well-formedness icon). Insert a second author element containing your name and email. Is this valid?
   

9. Insert a <Phone> tag, your phone number, and </Phone> after your email element. Is this valid?

10. Open the XML DTD in the editor. Try to figure out how you could have answered the previous two questions by examining the DTD rather than by experimentation.

11. Specify a style transformation for the XML instance by inserting as the third line of the instance

12. Open the XML instance in a browser again (in XML Spy, you can do this by clicking the “Browser” button at the bottom of the editor pane; in oXygen, click the red-triangle-in-a-circle icon to the right of the well-formedness icon).
    It should be formatted this time.
    

13. Delete the DTD specification. Does the style transform still work?
    What does this imply about XML transformation programs?

14. Open the XML transformation file (Report.xsl) in the editor.
    The third line of the program (where “xsl:template” occurs) matches the element named “Report” in the instance and then passes through as output everything up to the next “xsl:template” tag.

    Can you see how these 20 lines or so create the HTML “scaffold” for the formatted report?

Part 1 (4 points):

Rename a copy of Report.xml to YourlastnameA4.xml (e.g., GlushkoA4.xml).
Change the Author information to your own.

Note: You will have another section for your reflections from Part 2, listed below.

In the Body section of the report, change the Section title to “Reflections on Assignment 4” and write a short paragraph which:

• Describes (using 202 terms) what each of the three files we gave you does. (2 points)

• Explains the difference between an XML instance and the Document Type Definition. (1 point)

Validate your Report document and transform it to HTML. (1 point)

Part 2 (6 points):

Now that you have a feel for working in XML, we are asking you to return to A3 and encode the an instance of the vocabulary you submitted.

If during this process you wish to change your vocabulary, feel free, but the idea is that all the conceptual work you did in A3 should transfer over seamlessly. If you do make changes to your vocabulary, clearly describe your new vocabulary and descriptors in your reflection.

Please note: we are not asking you to create a DTD. Only an instance that follows the schema you laid out in A3. Put another way, pretend that the vocabulary you created in A3 is a DTD and follow it when creating your instance. Please name your instance YourNameA4Instance.xml. (3 points)

Next, create another Section in your Reflection report in Part 1 and title it “Reflections on Encoding Your Instance”. Write a paragraph or two (depends on how much you changed) describing your experience creating your instance. (3 points)

• Was it straightforward?

• Did it match the instance you created for A3?

• If not, what changed in the instance?

• If you had to make changes to your vocabulary, what were they and why did you decide to make them?

• Try to validate your document against Report.dtd. What happens and why?

Make sure your XML is still valid after you add this Section!

Extra Credit (up to 2 points):

Create a DTD for your vocabulary defining valid tags and attributes. Ensure that your instance validates against the DTD. Include the DTD in your submission zip with the name YourNameA4Vocabulary.dtd

Submission Instructions

Submit a total of five files (zipped). Name your zipped file YourNameA4.zip

You should include:

• YourNameA4Instance.xml

• YourNameA4Report.xml

• YourNameA4Report.dtd

• YourNameA4Report.xsl

• YourNameA4Report.html

• YourNameA4Vocabulary.dtd (extra credit)